libpq debug log

Started by Iwata, Ayaover 7 years ago233 messages
#1Iwata, Aya
iwata.aya@jp.fujitsu.com

Hi,

I'm going to propose libpq debug log for analysis of queries on the application side.
I think that it is useful to determine whether the cause is on the application side or the server side when a slow query occurs.

The provided information is "date and time" at which execution of processing is started, "query", "application side processing", which is picked up information from PQtrace(), and "connection id", which is for uniquely identifying the connection.

To collect the log, set the connection string or environment variable.
- logdir or PGLOGDIR : directory where log file written
- logsize or PGLOGSIZE : maximum log size

What do you think about this? Do you think that such feature is necessary?

Regards,
Aya Iwata

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Iwata, Aya (#1)
Re: libpq debug log

"Iwata, Aya" <iwata.aya@jp.fujitsu.com> writes:

I'm going to propose libpq debug log for analysis of queries on the application side.
I think that it is useful to determine whether the cause is on the application side or the server side when a slow query occurs.

Hm, how will you tell that really? And what's the advantage over the
existing server-side query logging capability?

The provided information is "date and time" at which execution of processing is started, "query", "application side processing", which is picked up information from PQtrace(), and "connection id", which is for uniquely identifying the connection.

PQtrace() is utterly useless for anything except debugging libpq
internals, and it's not tremendously useful even for that. Don't
bother with that part.

Where will you get a "unique connection id" from?

How will you deal with asynchronously-executed queries --- either
the PQgetResult style, or the single-row-at-a-time style?

regards, tom lane

#3Yugo Nagata
nagata@sraoss.co.jp
In reply to: Iwata, Aya (#1)
Re: libpq debug log

On Fri, 24 Aug 2018 04:38:22 +0000
"Iwata, Aya" <iwata.aya@jp.fujitsu.com> wrote:

Hi,

I'm going to propose libpq debug log for analysis of queries on the application side.
I think that it is useful to determine whether the cause is on the application side or the server side when a slow query occurs.

Do you mean you want to monitor the protocol message exchange at
the client side to analyze performance issues, right? Actually,
this might be useful to determin where is the problem, for example,
the client application, the backend PostgreSQL, or somewhere in the
network.

Such logging can be implemented in the application, but if libpq
provides the standard way, it would be helpful to resolve a problem
without modifying the application code.

The provided information is "date and time" at which execution of processing is started, "query", "application side processing", which is picked up information from PQtrace(), and "connection id", which is for uniquely identifying the connection.

I couldn't image how this is like. Could you show us a sample of log lines
in your head?

To collect the log, set the connection string or environment variable.
- logdir or PGLOGDIR : directory where log file written
- logsize or PGLOGSIZE : maximum log size

How we can specify the log file name? What should happen if a file size
exceeds to PGLOGSIZE?

Regards,
--
Yugo Nagata <nagata@sraoss.co.jp>

#4Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#2)
Re: libpq debug log

On Fri, Aug 24, 2018 at 9:48 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

PQtrace() is utterly useless for anything except debugging libpq
internals, and it's not tremendously useful even for that. Don't
bother with that part.

I think that improving the output format could help with that a lot.
What it current produces is almost unreadable; adjusting it to emit
one line per protocol message would, I think, help a lot. There are
probably other improvements that could be made at the same time.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#4)
Re: libpq debug log

Robert Haas <robertmhaas@gmail.com> writes:

On Fri, Aug 24, 2018 at 9:48 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

PQtrace() is utterly useless for anything except debugging libpq
internals, and it's not tremendously useful even for that. Don't
bother with that part.

I think that improving the output format could help with that a lot.
What it current produces is almost unreadable; adjusting it to emit
one line per protocol message would, I think, help a lot. There are
probably other improvements that could be made at the same time.

I wouldn't mind throwing it out and reimplementing it ;-) ... tracing
at the logical message level rather than the byte level would help.
But still, I'm not really convinced that it has very much to do with
what you'd want in a user-level debug log.

Part of what's really bad about the PQtrace output is that in v2 protocol
the same output can be repeated several times as we try to parse a message
and conclude we don't have it all yet. I believe that problem is gone
in v3, but it may be hard to do a consistent redesign until we nuke libpq's
v2 support. Still, it might be past time for the latter, seeing that
we no longer promise pre-7.4 compatibility in either psql or pg_dump.

regards, tom lane

#6Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Tom Lane (#2)
RE: libpq debug log

"Iwata, Aya" <iwata.aya@jp.fujitsu.com> writes:

I'm going to propose libpq debug log for analysis of queries on the application

side.

I think that it is useful to determine whether the cause is on the application

side or the server side when a slow query occurs.

Hm, how will you tell that really? And what's the advantage over the existing
server-side query logging capability?

The log I would like to propose is used when the performance issue happen,
system administrator knows the process of application internally and check if there is any problem.
"debug" is not the correct description of the feature. The correct one should be "trace". Should I create another thread?

The provided information is "date and time" at which execution of processing

is started, "query", "application side processing", which is picked up
information from PQtrace(), and "connection id", which is for uniquely
identifying the connection.

PQtrace() is utterly useless for anything except debugging libpq internals,
and it's not tremendously useful even for that. Don't bother with that part.

My initial intention was to get only useful information from PQTrace () since it acquires a lot of information.
Is there another way to obtain application side information besides PQTrace() ?

Where will you get a "unique connection id" from?

When collecting trace log file in the application side,
I think it is necessary to identify different connection.
In order to do this, when new empty PQconn structure is created, new connection id is also created.
Then we output it in the trace log file for one application.

How will you deal with asynchronously-executed queries --- either the
PQgetResult style, or the single-row-at-a-time style?

In my understanding, PQgetResult style outputs logs of multiple result queries at once,
While the single-row-at-a-time style outputs log for each query. Is this correct?
I think PQgetResult style is better,
because this style traces the internal process of the application.

Regards,
Aya Iwata

#7Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Yugo Nagata (#3)
RE: libpq debug log

I'm going to propose libpq debug log for analysis of queries on the application

side.

I think that it is useful to determine whether the cause is on the application

side or the server side when a slow query occurs.

Do you mean you want to monitor the protocol message exchange at the client
side to analyze performance issues, right? Actually, this might be useful to
determin where is the problem, for example, the client application, the backend
PostgreSQL, or somewhere in the network.

Such logging can be implemented in the application, but if libpq provides the
standard way, it would be helpful to resolve a problem without modifying the
application code.

Since I'd like to monitor the information the server and the client exchange,
I think monitoring protocol messages is good.

When a slow query is occurs, we check this client side trace log.
The purpose of this log acquisition I thought is to identify where is the problem:
server side, application side or traffic.
And if the problem is in application side, checking the trace log to identify what is the problem.

The provided information is "date and time" at which execution of processing

is started, "query", "application side processing", which is picked up
information from PQtrace(), and "connection id", which is for uniquely
identifying the connection.

I couldn't image how this is like. Could you show us a sample of log lines in
your head?

I am roughly thinking as follows;

...
START : 2018/09/03 18:16:35.357 CONNECTION(1)
STATUS : Connection
SEND MESSAGE : 2018/09/03 18:16:35.358
<information send to backend...>
RECEIVE MESSAGE : 2018/09/03 18:16:35.359
<information receive from backend...>
END : 2018/09/03 18:16:35.360
...
START : 2018/09/03 18:16:35.357 CONNECTION(1)
QUERY : DECLARE myportal CURSOR FOR select * from pg_database
SEND MESSAGE : 2018/09/03 18:16:35.358
<information send to backend...>
RECEIVE MESSAGE : 2018/09/03 18:16:35.359
<information receive from backend...>
END : 2018/09/03 18:16:35.360
...

To collect the log, set the connection string or environment variable.
- logdir or PGLOGDIR : directory where log file written
- logsize or PGLOGSIZE : maximum log size

How we can specify the log file name? What should happen if a file size exceeds
to PGLOGSIZE?

The log file name is determined as follow.
libpq-%ApplicationName-%Y-%m-%d_%H%M%S.log

When the log file size exceeds to PGLOGSIZE, the log is output to another file.

Regards,
Aya Iwata

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Iwata, Aya (#7)
Re: libpq debug log

"Iwata, Aya" <iwata.aya@jp.fujitsu.com> writes:

The purpose of this log acquisition I thought is to identify where is the problem:
server side, application side or traffic.

TBH, I think the sort of logging you're proposing would be expensive
enough that *it* would be the bottleneck in a lot of cases. A lot
of people find that the existing server-side "log_statement" support
is too expensive to keep turned on in production --- and that logs only
received SQL queries, not the returned data, and certainly not every
message passed over the wire.

regards, tom lane

#9Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Iwata, Aya (#7)
Re: libpq debug log

On 04/09/2018 02:29, Iwata, Aya wrote:

Since I'd like to monitor the information the server and the client exchange,
I think monitoring protocol messages is good.

When a slow query is occurs, we check this client side trace log.
The purpose of this log acquisition I thought is to identify where is the problem:
server side, application side or traffic.
And if the problem is in application side, checking the trace log to identify what is the problem.

Between perf/systemtap/dtrace and wireshark, you can already do pretty
much all of that. Have you looked at those and found anything missing?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#10Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Tom Lane (#8)
RE: libpq debug log

Let me explain this trace log in a bit more detail.

This log is not turned on in the system by default.
Turn it on when an issue occurs and you want to check the information in the application in order to clarify the cause.

I will present three use cases for this log.

1. Confirmation on cause of out-of-memory
We assume that Out-of-memory occurred in the process of storing the data received from the database.
However, the contents or length of the data is not known.
A trace log is obtained to find these out and what kind of data you have in which part (i.e. query result when receiving from database).

2. Protocol error confirmation
When there is an error in the protocol transmitted from the client and an error occurs in the database server, the protocol sent by the client is checked.
When the network is unstable, log is checked whether the database server is receiving protocol messages.

3. Processing delay survey
If the processing in the application is slow and the processing in the database is fast, investigate the cause of the processing delay.
4 kind of time can be obtained by the log;

Timestamp when SQL started
Timestamp when information began to be sent to the backend
Timestamp when information is successfully received in the application
Timestamp when SQL ended

Then the difference between the time is checked to find out which part of process takes time.

I reconfirm the function I proposed.

If get the trace log, set PGLOGDIR/logdir and PGLOGSIZE/logsize.
These parameters are set in the environment variable or the connection service file.
- logdir or PGLOGDIR : directory where log file written
- logsize or PGLOGSIZE : maximum log size. When the log file size exceeds to PGLOGSIZE, the log is output to another file.

The log file name is determined as follow.
libpq-%ConnectionID-%Y-%m-%d_%H%M%S.log

This is a trace log example;

...
Start: 2018-09-03 18:16:35.357 Connection(1)
Status: Connection
Send message: 2018-09-03 18:16:35.358
<information send to backend...>
Receive message: 2018-09-03 18:16:35.359
<information receive from backend...>
End: 2018-09-03 18:16:35.360
...
Start: 2018-09-03 18:16:35.357 Connection(1) ...(1), (2)
Query: DECLARE myportal CURSOR FOR select * from pg_database ...(3)
Send message: 2018-09-03 18:16:35.358 ...(4)
<information send to backend...> ...(5)
Receive message: 2018/09/03 18:16:35.359 ...(6)
<information receive from backend...> ...(7)
End: 2018-09-03 18:16:35.360 ...(8)
...

(1) Timestamp when SQL started
(2) Connection ID (Identify the connection)
(3) SQL statement
(4) Timestamp when information began to be sent to the backend
(5) send message to backend (Result of query, Protocol messages)
(6) Timestamp when information is successfully received in the application
(7) receive message from backend (Result of query, Protocol messages)
(8) Timestamp when SQL ended

Regards,
Iwata Aya

#11Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Peter Eisentraut (#9)
RE: libpq debug log

Hi,

Sorry for my late response.

Between perf/systemtap/dtrace and wireshark, you can already do pretty much
all of that. Have you looked at those and found anything missing?

These commands provide detailed information to us.
However, I think the trace log is necessary from the following point.

1. ease of use for users
It is necessary to take information that is easy to understand for database users.
This trace log is retrieved on the application server side.
Not only the database developer but also application developer will get and check this log.
Also, some of these commands return detailed PostgreSQL function names.
The trace log would be useful for users who do not know the inside of PostgreSQL (e.g. application developers)

2. obtain the same information on all OS
Some commands depend on the OS.
I think that it is important that the trace log information is compatible to each OS.

Regards,
Aya Iwata

#12Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#11)
1 attachment(s)
RE: libpq debug log

Hi,

I create a first libpq trace log patch.

In this patch,
- All message that PQtrace() gets are output to the libpq trace log file
(I maybe select more effective message in the future patch)
- Trace log output style is changed slightly from previously proposed

This patch not include documentation,
but you can see parameter detail and how to use it by looking at my previous e-mail.

If get the trace log, set PGLOGDIR/logdir and PGLOGSIZE/logsize.
These parameters are set in the environment variable or the connection service
file.
- logdir or PGLOGDIR : directory where log file written
- logsize or PGLOGSIZE : maximum log size(M). When the log file size exceeds to
PGLOGSIZE, the log is output to another file.

The log file name is determined as follow.
libpq-%ProcessID-%Y-%m-%d_%H%M%S.log

Trace log example;
Start : 2018/10/30 08:02:24.433 ... time(a)
Query: SELECT pg_catalog.set_config('search_path', '', false)
To backend> Msg Q
To backend> "SELECT pg_catalog.set_config('search_path', '', false)"
To backend> Msg complete, length 60
Start sending message to backend: 2018/10/30 08:02:24.433 ... time(b)
End sending message to backend: 2018/10/30 08:02:24.433 ... time(c)
Start receiving message from backend: 2018/10/30 08:02:24.434 ... time(d)
End receiving message from backend: 2018/10/30 08:02:24.434 ... time(e)
From backend> T
From backend (#4)> 35
From backend (#2)> 1
From backend> "set_config"
From backend (#4)> 0
From backend (#2)> 0
From backend (#4)> 25
From backend (#2)> 65535
From backend (#4)> -1
From backend (#2)> 0
From backend> D
From backend (#4)> 10
From backend (#2)> 1
From backend (#4)> 0
From backend> C
From backend (#4)> 13
From backend> "SELECT 1"
From backend> Z
From backend (#4)> 5
From backend> Z
From backend (#4)> 5
From backend> I
End : 2018/10/30 08:02:24.435 ... time(f)

From time(a) to time(b): time for libpq processing
From time(b) to time(c): time for traffic
From time(c) to time(d): time for backend processing
From time(d) to time(e): time for traffic
From time(e) to time(f): time for libpq processing

Regards,
Aya Iwata

Attachments:

v1-0001-libpq-trace-log.patchapplication/octet-stream; name=v1-0001-libpq-trace-log.patchDownload
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d001bc5..6bdf2df 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -325,6 +325,15 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1137,19 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3687,6 +3709,14 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3722,6 +3752,8 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		if (conn->Pfdebug || conn->traceDebug)
+			traceLog_fprintf(conn, true, "Send connection terminate message to backend: ");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4124,6 +4156,9 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	if (conn->Pfdebug || conn->traceDebug)
+		traceLog_fprintf(conn, true, "Send connection start message to backend: ");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 6aed8c8..d326e66 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,9 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	if (conn->Pfdebug || conn->traceDebug)
+		traceLog_fprintf(conn, false, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1219,6 +1219,9 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Query: %s \n",query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1337,6 +1340,9 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Statement name: %s, Query: %s \n",stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1507,6 +1513,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		if (conn->traceDebug)
+			traceLog_fprintf(conn, false, "Statement name: %s, Command: %s \n",stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1820,6 +1829,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -2006,6 +2016,9 @@ PQexecStart(PGconn *conn)
 {
 	PGresult   *result;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start :");
+
 	if (!conn)
 		return false;
 
@@ -2120,6 +2133,9 @@ PQexecFinish(PGconn *conn)
 			break;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End :");
+
 	return lastResult;
 }
 
@@ -2218,6 +2234,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Describe type: %c, target: %s \n",desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 46ece1a..fcf7c8e 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,168 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn,char* filename);
+static void traceLog_fputnbytes(PGconn *conn, const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime,int type);
+#define	TRACELOG_TIME_SIZE	28
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime,int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond);
+	else if(type==1)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond,
+				localTime.wMilliseconds);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm);
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn,char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime,0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#endif
+}
+
+/*
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn,logfilename);
+	conn->traceDebug=fopen(logfilename,"w");
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, bool addTime, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret;
+	char		currenttime[TRACELOG_TIME_SIZE];
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	if(conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+	else if(conn->traceDebug)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+
+		/* Select trace log message style */
+		if(addTime)
+		{
+			getCurrentTime(currenttime,1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", msgBuf, currenttime);
+		}
+		else
+		{
+			ret = fprintf(conn->traceDebug, "%s",msgBuf);
+		}
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn,const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug,str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug,str,n);
+		fprintf(conn->traceDebug,"\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +262,8 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +278,8 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +316,9 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +345,8 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +358,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,11 +367,10 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
+	if(conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,s, len);
 	}
 
 	return 0;
@@ -228,7 +392,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 
 	if (conn->Pfdebug)
 	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
+		fprintf(conn->Pfdebug, "(pqSkipnchar)From backend (%lu)> ", (unsigned long) len);
 		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
 		fprintf(conn->Pfdebug, "\n");
 	}
@@ -245,14 +409,14 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
+	if(conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf,"To backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,s,len);
 	}
 
 	return 0;
@@ -292,8 +456,8 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +492,8 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +712,9 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +750,9 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +841,16 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start receiving message from backend:");
 retry3:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +938,16 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start receiving message from backend:");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1024,9 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start sending message to backend:");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1039,9 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End sending message to backend:");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 975ab33..2aa8a51 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -499,6 +499,11 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	int			logsize;		/* Trace log maximum size */
+	char	   *logdir;			/* Trace log directory to save log */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -665,6 +670,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, bool addTime, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#13Jim Doty
jdoty@pivotal.io
In reply to: Iwata, Aya (#12)
Re: libpq debug log

Greetings,

This is my first attempt at a patch review, so I will take a pass at the
low hanging fruit.

Initial Pass
============

+ Patch applies
+ Patch builds
+ Patch behaves as described in the thread

I tried a few small things:

When I set a relative path for `PGLOGDIR`, the files were correctly
written to the directory.

When I set a path for `PGLOGDIR` that didn't exist or was not
write-able, the patch writes no files, and does not alert the user that
no files are being written.

Performance
===========

I ran two permutations of make check, one with the patch applied but not
activated, and the other with with the files being written to disk. Each
permutation was run ten times, and the stats are below (times are in
seconds):

min max median mean
not logging 50.4 57.6 53.3 53.4
logging 58.3 77.7 65.0 65.8

Cheers,
Jim Doty

#14Jacob Champion
pchampion@pivotal.io
In reply to: Iwata, Aya (#12)
Re: libpq debug log

On Tue, Oct 30, 2018 at 2:39 AM Iwata, Aya <iwata.aya@jp.fujitsu.com> wrote:

I create a first libpq trace log patch.

Couple additional thoughts from a read-through of the patch:

- PQtrace() and the new trace-logging machinery overlap in some places
but not others -- and if both are set, PQtrace() will take precedence.
It seems like the two should not be separate.
- It isn't immediately clear to me how the new information in the logs
is intended to be used in concert with the old information. Maybe this
goes back to the comments by Tom and Robert higher in the thread --
that an overhaul of the PQtrace system is due. This patch as presented
would make things a little worse before they got better, I think.

That said, I think the overall idea -- application performance
information that can be enabled via the environment, without requiring
debugging privileges on a machine or the need to manually correlate
traces made by other applications -- is a good one, and something I
would use.

--Jacob

#15Haribabu Kommi
kommi.haribabu@gmail.com
In reply to: Iwata, Aya (#12)
Re: libpq debug log

On Tue, Oct 30, 2018 at 8:38 PM Iwata, Aya <iwata.aya@jp.fujitsu.com> wrote:

Hi,

I create a first libpq trace log patch.

In this patch,
- All message that PQtrace() gets are output to the libpq trace log file
(I maybe select more effective message in the future patch)
- Trace log output style is changed slightly from previously proposed

This patch not include documentation,
but you can see parameter detail and how to use it by looking at my
previous e-mail.

If get the trace log, set PGLOGDIR/logdir and PGLOGSIZE/logsize.
These parameters are set in the environment variable or the connection
service
file.
- logdir or PGLOGDIR : directory where log file written
- logsize or PGLOGSIZE : maximum log size(M). When the log file size
exceeds to
PGLOGSIZE, the log is output to another file.

The log file name is determined as follow.
libpq-%ProcessID-%Y-%m-%d_%H%M%S.log

Trace log example;
Start : 2018/10/30 08:02:24.433
... time(a)
Query: SELECT pg_catalog.set_config('search_path', '', false)
To backend> Msg Q
To backend> "SELECT pg_catalog.set_config('search_path', '', false)"
To backend> Msg complete, length 60
Start sending message to backend: 2018/10/30 08:02:24.433
... time(b)
End sending message to backend: 2018/10/30 08:02:24.433
... time(c)
Start receiving message from backend: 2018/10/30 08:02:24.434 ... time(d)
End receiving message from backend: 2018/10/30 08:02:24.434 ... time(e)
From backend> T
From backend (#4)> 35
From backend (#2)> 1
From backend> "set_config"
From backend (#4)> 0
From backend (#2)> 0
From backend (#4)> 25
From backend (#2)> 65535
From backend (#4)> -1
From backend (#2)> 0
From backend> D
From backend (#4)> 10
From backend (#2)> 1
From backend (#4)> 0
From backend> C
From backend (#4)> 13
From backend> "SELECT 1"
From backend> Z
From backend (#4)> 5
From backend> Z
From backend (#4)> 5
From backend> I
End : 2018/10/30 08:02:24.435 ... time(f)

From time(a) to time(b): time for libpq processing
From time(b) to time(c): time for traffic
From time(c) to time(d): time for backend processing
From time(d) to time(e): time for traffic
From time(e) to time(f): time for libpq processing

Thanks for the patch.

I have some comments related to the trace output that is getting
printed. The amount of log it is generating may not be understood
to many of the application developers. IMO, this should print
only the necessary information that can understood by any one
by default and full log with more configuration?

Regards,
Haribabu Kommi
Fujitsu Australia

#16Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Jim Doty (#13)
1 attachment(s)
RE: libpq debug log

Hi Jim Doty san,

Thank you for review! I'm sorry my late reply...

Initial Pass
============

+ Patch applies
+ Patch builds
+ Patch behaves as described in the thread

Thank you for your check.

When I set a path for `PGLOGDIR` that didn't exist or was not write-able,
the patch writes no files, and does not alert the user that no files are being
written.

I understand. I think it means that it is necessary to confirm how the setting is going well.
There is no warning method when connection string or the environment variable is wrong.

So I added following document:
+   If the setting of the file path by the connection string or the environment variable is
+   incorrect, the log file is not created in the intended location.
+   The maximum log file size you set is output to the beginning of the file, so you can check it.
And I added the process. Please see my v2 patch.

Performance
===========

I ran two permutations of make check, one with the patch applied but not
activated, and the other with with the files being written to disk. Each
permutation was run ten times, and the stats are below (times are in
seconds):

min max median mean
not logging 50.4 57.6 53.3 53.4
logging 58.3 77.7 65.0 65.8

Thank you for your measurement.
I'm thinking about adding a logging level so that only the necessary information can be printed by default. It was pointed out by Haribabu san's e-mail.
This minimizes the impact of logging on performance.

Regards,
Aya Iwata

Attachments:

v2-0001-libpq-trace-log.patchapplication/octet-stream; name=v2-0001-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d2e5b08..0d46d38 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1589,6 +1589,28 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </para>
       </listitem>
     </varlistentry>
+
+     <varlistentry id="libpq-connect-log-dir" xreflabel="logdir">
+      <term><literal>logdir</literal></term>
+      <listitem>
+       <para>
+        Path of directory where log file written. When both <literal>logdir</literal>
+        and <literal>logsize</literal> is set, logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-size" xreflabel="logsize">
+      <term><literal>logsize</literal></term>
+      <listitem>
+       <para>
+        Maximum log size. The unit is megabyte. When the log file size exceeds 
+        to <literal>logsize</literal>, the log is output to another file. 
+        When both <literal>logdir</literal> and <literal>logsize</literal> is set, 
+        logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
     </variablelist>
    </para>
   </sect2>
@@ -7472,6 +7494,26 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
       linkend="libpq-connect-target-session-attrs"/> connection parameter.
      </para>
     </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGDIR</envar></primary>
+      </indexterm>
+      <envar>PGLOGDIR</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-dir"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGSIZE</envar></primary>
+      </indexterm>
+      <envar>PGLOGSIZE</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-size"/> connection parameter.
+     </para>
+    </listitem>
    </itemizedlist>
   </para>
 
@@ -8300,6 +8342,35 @@ int PQisthreadsafe();
   </para>
  </sect1>
 
+ <sect1 id="libpq-logging">
+  <title>Logging</title>
+
+  <indexterm zone="libpq-logging">
+   <primary>trace log</primary>
+  </indexterm>
+  <indexterm zone="libpq-logging">
+   <primary>logging</primary>
+  </indexterm>
+
+  <para>
+   Libpq trace log can trace information about SQL queries which issued by 
+   the libpq application. This output help determine whether the couse is 
+   on backend side or application side and solve issues with the libpq Driver 
+   when it use. The log without requiring recompile of the libpq application.
+  </para>
+
+  <para>
+   You can activate this log by setting both <parameter>logdir</parameter> and 
+   <parameter>logsize</parameter> of the connection string, or by setting both 
+   the environment variables <envar>PGLOGDIR</envar> and <envar>PGLOGSIZE</envar>.
+   The log trace file is written in a directory setting by <envar>PGLOGDIR</envar> 
+   or <parameter>logdir</parameter>.
+   The file name is determined as <filename>libpq-%ProcessID-%Y-%m-%d_%H%M%S.log</filename>.
+   If the setting of the file path by the connection string or the environment variable is 
+   incorrect, the log file is not created in the intended location. 
+   The maximum log file size you set is output to the beginning of the file, so you can check it.
+  </para>
+ </sect1>
 
  <sect1 id="libpq-build">
   <title>Building <application>libpq</application> Programs</title>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bc456fe..67c1f12 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -325,6 +325,15 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1137,19 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3716,6 +3738,14 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3751,6 +3781,8 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		if (conn->Pfdebug || conn->traceDebug)
+			traceLog_fprintf(conn, true, "Send connection terminate message to backend: ");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4153,6 +4185,9 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	if (conn->Pfdebug || conn->traceDebug)
+		traceLog_fprintf(conn, true, "Send connection start message to backend: ");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 6aed8c8..675d1bd 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,9 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	if (conn->Pfdebug || conn->traceDebug)
+		traceLog_fprintf(conn, false, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1208,6 +1208,9 @@ fail:
 int
 PQsendQuery(PGconn *conn, const char *query)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQuery start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1219,6 +1222,9 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Query: %s \n",query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1249,6 +1255,10 @@ PQsendQuery(PGconn *conn, const char *query)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQuery end :");
+
 	return 1;
 }
 
@@ -1266,6 +1276,9 @@ PQsendQueryParams(PGconn *conn,
 				  const int *paramFormats,
 				  int resultFormat)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryParams start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1283,6 +1296,9 @@ PQsendQueryParams(PGconn *conn,
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryParams end :");
+
 	return PQsendQueryGuts(conn,
 						   command,
 						   "",	/* use unnamed statement */
@@ -1306,6 +1322,9 @@ PQsendPrepare(PGconn *conn,
 			  const char *stmtName, const char *query,
 			  int nParams, const Oid *paramTypes)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendPrepare start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1337,6 +1356,9 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Statement name: %s, Query: %s \n",stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1386,6 +1408,10 @@ PQsendPrepare(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendPrepare end :");
+
 	return 1;
 
 sendFailed:
@@ -1492,6 +1518,9 @@ PQsendQueryGuts(PGconn *conn,
 {
 	int			i;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryGuts start :");
+
 	/* This isn't gonna work on a 2.0 server */
 	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 	{
@@ -1507,6 +1536,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		if (conn->traceDebug)
+			traceLog_fprintf(conn, false, "Statement name: %s, Command: %s \n",stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1638,6 +1670,10 @@ PQsendQueryGuts(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryGuts end :");
+
 	return 1;
 
 sendFailed:
@@ -1820,6 +1856,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -2081,6 +2118,9 @@ PQexecFinish(PGconn *conn)
 	PGresult   *result;
 	PGresult   *lastResult;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQexecFinish start :");
+
 	/*
 	 * For backwards compatibility, return the last result if there are more
 	 * than one --- but merge error messages if we get more than one error
@@ -2120,6 +2160,9 @@ PQexecFinish(PGconn *conn)
 			break;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQexecFinish end :");
+
 	return lastResult;
 }
 
@@ -2203,6 +2246,9 @@ PQsendDescribePortal(PGconn *conn, const char *portal)
 static int
 PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendDescribe start :");
+
 	/* Treat null desc_target as empty string */
 	if (!desc_target)
 		desc_target = "";
@@ -2218,6 +2264,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Describe type: %c, target: %s \n",desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
@@ -2249,6 +2298,10 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendDescribe end :");
+
 	return 1;
 
 sendFailed:
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 46ece1a..ba60acb 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,169 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn,char* filename);
+static void traceLog_fputnbytes(PGconn *conn, const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime,int type);
+#define	TRACELOG_TIME_SIZE	28
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime,int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond);
+	else if(type==1)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond,
+				localTime.wMilliseconds);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm);
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn,char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime,0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#endif
+}
+
+/* 
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn,logfilename);
+	conn->traceDebug=fopen(logfilename,"w");
+	fprintf(conn->traceDebug, "Maximum log size is %d B.\n", conn->logsize);
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, bool addTime, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret;
+	char		currenttime[TRACELOG_TIME_SIZE];
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	if(conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+	else if(conn->traceDebug)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+
+		/* Select trace log message style */
+		if(addTime)
+		{
+			getCurrentTime(currenttime,1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", msgBuf, currenttime);
+		}
+		else
+		{
+			ret = fprintf(conn->traceDebug, "%s",msgBuf);
+		}
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn,const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug,str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug,str,n);
+		fprintf(conn->traceDebug,"\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +263,8 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +279,8 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +317,9 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +346,8 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +359,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,11 +368,10 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
+	if(conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,s, len);
 	}
 
 	return 0;
@@ -228,7 +393,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 
 	if (conn->Pfdebug)
 	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
+		fprintf(conn->Pfdebug, "(pqSkipnchar)From backend (%lu)> ", (unsigned long) len);
 		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
 		fprintf(conn->Pfdebug, "\n");
 	}
@@ -245,14 +410,14 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
+	if(conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf,"To backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,s,len);
 	}
 
 	return 0;
@@ -292,8 +457,8 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +493,8 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +713,9 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +751,9 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +842,17 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
 retry3:
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start receiving message from backend:");
+
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +940,16 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start receiving message from backend:");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1026,9 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start sending message to backend:");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1041,9 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End sending message to backend:");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66fd317..71d4329 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -500,6 +500,11 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	int			logsize;		/* Trace log maximum size */
+	char	   *logdir;			/* Trace log directory to save log */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -666,6 +671,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, bool addTime, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#17Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Haribabu Kommi (#15)
RE: libpq debug log

Hi Hari san,

Thank you for your comment! And sorry my late reply…

I have some comments related to the trace output that is getting
printed. The amount of log it is generating may not be understood
to many of the application developers. IMO, this should print
only the necessary information that can understood by any one
by default and full log with more configuration?

I understand. And I agree your opinion.
I will add feature called “log level” that changes the amount of log output information with my future version patch.

Regards,
Aya Iwata

#18Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Jacob Champion (#14)
RE: libpq debug log

Hi Jacob san,

Thank you for your comment! And sorry for late reply...

Couple additional thoughts from a read-through of the patch:

- PQtrace() and the new trace-logging machinery overlap in some places but
not others -- and if both are set, PQtrace() will take precedence.
It seems like the two should not be separate.

I understand. This log is acquired for the purpose of investigating the cause part (server side or application side) when performance is bad.
So I will search whether getting other place of PQtrace() is necessary or not.
And I will reply after the research, please wait for a while a little.

- It isn't immediately clear to me how the new information in the logs is
intended to be used in concert with the old information. Maybe this goes back
to the comments by Tom and Robert higher in the thread -- that an overhaul
of the PQtrace system is due. This patch as presented would make things a
little worse before they got better, I think.

That said, I think the overall idea -- application performance information
that can be enabled via the environment, without requiring debugging
privileges on a machine or the need to manually correlate traces made by other
applications -- is a good one, and something I would use.

Thank you. I think so, too. Some applications cannot be stopped to add the PQtrace() code.

Regards,
Aya Iwata

#19Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#18)
1 attachment(s)
RE: libpq debug log

Hi,

I created a new version patch. Please find attached my patch.

Changes:
Since v2 patch
I forgot to write the detail of v2 patch changes. So I write these.
- Fixed the " Start receiving message from backend:" message location. Because I found that message located at outside of "retry4".
- Changed location where output "start :" / "end :" messages and timestamp. The reason of the change is that v1 patch did not correspond to Asynchronous Command Processing.
- Added documentation
- Added documentation how to check mistake of logdir and/or logsize. (Based on review comment of Jim san's)
Since v3 patch
- Fixed my mistake on fe-connect.c. Time and message were output at the place where does not output in originally PQtrace(). These messages are needed only new trace log. So I fixed it.
- Fixed two points so that new trace log overlaps all places in PQtrace(). (Based on review comment of Jacob san's)

TODO:
I will add the feature called "logging level" on next version patch. I will attach it as soon as possible.

I'm marking it as "Needs review".

Regards,
Aya Iwata

Attachments:

v3-0001-libpq-trace-log.patchapplication/octet-stream; name=v3-0001-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d2e5b08..0d46d38 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1589,6 +1589,28 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </para>
       </listitem>
     </varlistentry>
+
+     <varlistentry id="libpq-connect-log-dir" xreflabel="logdir">
+      <term><literal>logdir</literal></term>
+      <listitem>
+       <para>
+        Path of directory where log file written. When both <literal>logdir</literal>
+        and <literal>logsize</literal> is set, logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-size" xreflabel="logsize">
+      <term><literal>logsize</literal></term>
+      <listitem>
+       <para>
+        Maximum log size. The unit is megabyte. When the log file size exceeds 
+        to <literal>logsize</literal>, the log is output to another file. 
+        When both <literal>logdir</literal> and <literal>logsize</literal> is set, 
+        logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
     </variablelist>
    </para>
   </sect2>
@@ -7472,6 +7494,26 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
       linkend="libpq-connect-target-session-attrs"/> connection parameter.
      </para>
     </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGDIR</envar></primary>
+      </indexterm>
+      <envar>PGLOGDIR</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-dir"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGSIZE</envar></primary>
+      </indexterm>
+      <envar>PGLOGSIZE</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-size"/> connection parameter.
+     </para>
+    </listitem>
    </itemizedlist>
   </para>
 
@@ -8300,6 +8342,35 @@ int PQisthreadsafe();
   </para>
  </sect1>
 
+ <sect1 id="libpq-logging">
+  <title>Logging</title>
+
+  <indexterm zone="libpq-logging">
+   <primary>trace log</primary>
+  </indexterm>
+  <indexterm zone="libpq-logging">
+   <primary>logging</primary>
+  </indexterm>
+
+  <para>
+   Libpq trace log can trace information about SQL queries which issued by 
+   the libpq application. This output help determine whether the couse is 
+   on backend side or application side and solve issues with the libpq Driver 
+   when it use. The log without requiring recompile of the libpq application.
+  </para>
+
+  <para>
+   You can activate this log by setting both <parameter>logdir</parameter> and 
+   <parameter>logsize</parameter> of the connection string, or by setting both 
+   the environment variables <envar>PGLOGDIR</envar> and <envar>PGLOGSIZE</envar>.
+   The log trace file is written in a directory setting by <envar>PGLOGDIR</envar> 
+   or <parameter>logdir</parameter>.
+   The file name is determined as <filename>libpq-%ProcessID-%Y-%m-%d_%H%M%S.log</filename>.
+   If the setting of the file path by the connection string or the environment variable is 
+   incorrect, the log file is not created in the intended location. 
+   The maximum log file size you set is output to the beginning of the file, so you can check it.
+  </para>
+ </sect1>
 
  <sect1 id="libpq-build">
   <title>Building <application>libpq</application> Programs</title>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bc456fe..d6d16a0 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -325,6 +325,15 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1137,19 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3716,6 +3738,14 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3751,6 +3781,8 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		if (conn->traceDebug)
+			traceLog_fprintf(conn, true, "Send connection terminate message to backend: ");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4153,6 +4185,9 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Send connection start message to backend: ");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 6aed8c8..d45e795 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,9 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	if (conn->Pfdebug || conn->traceDebug)
+		traceLog_fprintf(conn, false, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1208,6 +1208,9 @@ fail:
 int
 PQsendQuery(PGconn *conn, const char *query)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQuery start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1219,6 +1222,9 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Query: %s \n",query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1249,6 +1255,10 @@ PQsendQuery(PGconn *conn, const char *query)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQuery end :");
+
 	return 1;
 }
 
@@ -1266,6 +1276,9 @@ PQsendQueryParams(PGconn *conn,
 				  const int *paramFormats,
 				  int resultFormat)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryParams start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1283,6 +1296,9 @@ PQsendQueryParams(PGconn *conn,
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryParams end :");
+
 	return PQsendQueryGuts(conn,
 						   command,
 						   "",	/* use unnamed statement */
@@ -1306,6 +1322,9 @@ PQsendPrepare(PGconn *conn,
 			  const char *stmtName, const char *query,
 			  int nParams, const Oid *paramTypes)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendPrepare start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1337,6 +1356,9 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Statement name: %s, Query: %s \n",stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1386,6 +1408,10 @@ PQsendPrepare(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendPrepare end :");
+
 	return 1;
 
 sendFailed:
@@ -1492,6 +1518,9 @@ PQsendQueryGuts(PGconn *conn,
 {
 	int			i;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryGuts start :");
+
 	/* This isn't gonna work on a 2.0 server */
 	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 	{
@@ -1507,6 +1536,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		if (conn->traceDebug)
+			traceLog_fprintf(conn, false, "Statement name: %s, Command: %s \n",stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1638,6 +1670,10 @@ PQsendQueryGuts(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendQueryGuts end :");
+
 	return 1;
 
 sendFailed:
@@ -1780,6 +1816,9 @@ PQgetResult(PGconn *conn)
 {
 	PGresult   *res;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQgetResult start :");
+
 	if (!conn)
 		return NULL;
 
@@ -1820,6 +1859,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -1874,6 +1914,9 @@ PQgetResult(PGconn *conn)
 		}
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQgetResult end :");
+
 	return res;
 }
 
@@ -2203,6 +2246,9 @@ PQsendDescribePortal(PGconn *conn, const char *portal)
 static int
 PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 {
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendDescribe start :");
+
 	/* Treat null desc_target as empty string */
 	if (!desc_target)
 		desc_target = "";
@@ -2218,6 +2264,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, false, "Describe type: %c, target: %s \n",desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
@@ -2249,6 +2298,10 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "PQsendDescribe end :");
+
 	return 1;
 
 sendFailed:
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 46ece1a..75c5040 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,169 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn,char* filename);
+static void traceLog_fputnbytes(PGconn *conn, const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime,int type);
+#define	TRACELOG_TIME_SIZE	28
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime,int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond);
+	else if(type==1)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond,
+				localTime.wMilliseconds);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm);
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn,char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime,0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#endif
+}
+
+/* 
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn,logfilename);
+	conn->traceDebug=fopen(logfilename,"w");
+	fprintf(conn->traceDebug, "Maximum log size is %d B.\n", conn->logsize);
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, bool addTime, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret;
+	char		currenttime[TRACELOG_TIME_SIZE];
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	if(conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+	else if(conn->traceDebug)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+
+		/* Select trace log message style */
+		if(addTime)
+		{
+			getCurrentTime(currenttime,1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", msgBuf, currenttime);
+		}
+		else
+		{
+			ret = fprintf(conn->traceDebug, "%s",msgBuf);
+		}
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn,const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug,str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug,str,n);
+		fprintf(conn->traceDebug,"\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +263,8 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +279,8 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +317,9 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +346,8 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +359,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,11 +368,10 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
+	if(conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,s, len);
 	}
 
 	return 0;
@@ -223,14 +388,14 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 int
 pqSkipnchar(size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
+	if (conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,conn->inBuffer + conn->inCursor, len);
 	}
 
 	conn->inCursor += len;
@@ -245,14 +410,14 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
+	if(conn->Pfdebug||conn->traceDebug)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
+		sprintf(buf,"To backend (%lu)> ", (unsigned long) len);
+		traceLog_fputnbytes(conn,buf,s,len);
 	}
 
 	return 0;
@@ -292,8 +457,8 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +493,8 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +713,9 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +751,9 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	if(conn->Pfdebug||conn->traceDebug)
+		traceLog_fprintf(conn, false, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +842,17 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
 retry3:
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start receiving message from backend:");
+
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +940,16 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start receiving message from backend:");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1026,9 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "Start sending message to backend:");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1041,9 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	if (conn->traceDebug)
+		traceLog_fprintf(conn, true, "End sending message to backend:");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
@@ -963,6 +1149,9 @@ pqFlush(PGconn *conn)
 	if (conn->Pfdebug)
 		fflush(conn->Pfdebug);
 
+	if (conn->traceDebug)
+		fflush(conn->traceDebug);
+
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
 
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66fd317..71d4329 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -500,6 +500,11 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	int			logsize;		/* Trace log maximum size */
+	char	   *logdir;			/* Trace log directory to save log */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -666,6 +671,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, bool addTime, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#20Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Iwata, Aya (#19)
Re: libpq debug log

On 27/11/2018 08:42, Iwata, Aya wrote:

I created a new version patch. Please find attached my patch.

This does not excite me. It seems mostly redundant with using tcpdump.

If I were to debug networking problems, I wouldn't even trust this very
much because it relies on the willpower of all future PostgreSQL
developers to keep this accurately up to date, whereas tcpdump gives me
the truth from the kernel.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#21Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Peter Eisentraut (#20)
RE: libpq debug log

Hi Peter,

Thank you for your reply!

On 27/11/2018 08:42, Iwata, Aya wrote:

I created a new version patch. Please find attached my patch.

This does not excite me. It seems mostly redundant with using tcpdump.

I will develop "log level". I'm planning not to output redundant message at the default level.

If I were to debug networking problems, I wouldn't even trust this very much
because it relies on the willpower of all future PostgreSQL developers to
keep this accurately up to date, whereas tcpdump gives me the truth from the
kernel.

I agree your concern about log trusty. It would be a good choice for only
skilled users to use tcpdump. I think libpq trace log will be used many users,
it includes users who not familiar with PostgreSQL protocols. The log would be easier to use
because it shows "start time" and "end time". On tcpdump also shows
the starting time and ending time but people need to know PostgreSQL protocol
to get them.

And this log also is useful for Windows users.
Windows does not have originally networking trace tool.

If you have any ideas about maintain this feature, I would like to know it.

Regards,
Aya Iwata

#22Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#21)
1 attachment(s)
RE: libpq debug log

Hi,

TODO:
I will add the feature called "logging level" on next version patch. I will
attach it as soon as possible.

I created v4 patch. Please find attached the patch.
This patch developed "logminlevel" parameter. level1 and level2 can be set, level1 is the default.
If level1 is set, it outputs the time in the functions. It helps to find out where it takes time.
If level2 is set, it outputs information on the protocol being exchanged in addition to level1 information.

I would appreciate if you could review my latest patch.

Regards,
Aya Iwata

Attachments:

v4-0001-libpq-trace-log.patchapplication/octet-stream; name=v4-0001-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d2e5b08..080d24c 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1589,6 +1589,42 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </para>
       </listitem>
     </varlistentry>
+
+     <varlistentry id="libpq-connect-log-dir" xreflabel="logdir">
+      <term><literal>logdir</literal></term>
+      <listitem>
+       <para>
+        Path of directory where log file written. When both <literal>logdir</literal>
+        and <literal>logsize</literal> is set, logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-size" xreflabel="logsize">
+      <term><literal>logsize</literal></term>
+      <listitem>
+       <para>
+        Maximum log size. The unit is megabyte. When the log file size exceeds 
+        to <literal>logsize</literal>, the log is output to another file. 
+        When both <literal>logdir</literal> and <literal>logsize</literal> is set, 
+        logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-min-level" xreflabel="logminlevel">
+      <term><literal>logminlevel</literal></term>
+      <listitem>
+       <para>
+        Level of the log. <literal>level1</literal> and <literal>level2</literal> 
+        can be set, <literal>level1</literal> is the default. 
+        When this parameter is <literal>level1</literal>,it outputs the time 
+        in the function and connection time. <literal>level2</literal> outputs 
+        information on the protocol being exchanged in addition to 
+        <literal>level1</literal> information.
+      </para>
+     </listitem>
+    </varlistentry>
     </variablelist>
    </para>
   </sect2>
@@ -7472,6 +7508,36 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
       linkend="libpq-connect-target-session-attrs"/> connection parameter.
      </para>
     </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGDIR</envar></primary>
+      </indexterm>
+      <envar>PGLOGDIR</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-dir"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGSIZE</envar></primary>
+      </indexterm>
+      <envar>PGLOGSIZE</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-size"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGMINLEVEL</envar></primary>
+      </indexterm>
+      <envar>PGLOGMINLEVEL</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-min-level"/> connection parameter.
+     </para>
+    </listitem>
    </itemizedlist>
   </para>
 
@@ -8300,6 +8366,38 @@ int PQisthreadsafe();
   </para>
  </sect1>
 
+ <sect1 id="libpq-logging">
+  <title>Logging</title>
+
+  <indexterm zone="libpq-logging">
+   <primary>trace log</primary>
+  </indexterm>
+  <indexterm zone="libpq-logging">
+   <primary>logging</primary>
+  </indexterm>
+
+  <para>
+   Libpq trace log can trace information about SQL queries which issued by 
+   the libpq application. This output help determine whether the couse is 
+   on backend side or application side and solve issues with the libpq Driver 
+   when it use. The log without requiring recompile of the libpq application.
+  </para>
+
+  <para>
+   You can activate this log by setting both <parameter>logdir</parameter> and 
+   <parameter>logsize</parameter> of the connection string, or by setting both 
+   the environment variables <envar>PGLOGDIR</envar> and <envar>PGLOGSIZE</envar>.
+   The log trace file is written in a directory setting by <envar>PGLOGDIR</envar> 
+   or <parameter>logdir</parameter>.
+   Information to be output to the log file can be controlled by setting 
+   <parameter>logminlevel</parameter> or <envar>PGLOGMINLEVEL</envar>.
+   The file name is determined as <filename>libpq-%ProcessID-%Y-%m-%d_%H%M%S.log</filename>.
+   If the setting of the file path by the connection string or the environment variable is 
+   incorrect, the log file is not created in the intended location. 
+   The maximum log file size and log level you set is output to the beginning of the file, 
+   so you can check it.
+  </para>
+ </sect1>
 
  <sect1 id="libpq-build">
   <title>Building <application>libpq</application> Programs</title>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bc456fe..2b441be 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -128,6 +128,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
 #define DefaultSSLMode "prefer"
 #else
 #define DefaultSSLMode	"disable"
+#define DefaultLogMinLevel		"level1"
 #endif
 
 /* ----------
@@ -325,6 +326,19 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
+	{"logminlevel", "PGLOGMINLEVEL", DefaultLogMinLevel, NULL,
+		"LogMinlevel", "", 7,
+	offsetof(struct pg_conn, logminlevel_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1142,26 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+
+		if(strcmp(conn->logminlevel_str, "level1") == 0)
+			conn->logminlevel = LEVEL1;
+
+		if(strcmp(conn->logminlevel_str, "level2") == 0)
+			conn->logminlevel = LEVEL2;
+
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3716,6 +3750,16 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->logminlevel_str)
+		free(conn->logminlevel_str);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3751,6 +3795,7 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Send connection terminate message to backend: ");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4153,6 +4198,8 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	traceLog_fprintf(conn, LEVEL1, "Send connection start message to backend: ");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 6aed8c8..42279a6 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,8 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	traceLog_fprintf(conn, LEVEL2, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1208,6 +1207,8 @@ fail:
 int
 PQsendQuery(PGconn *conn, const char *query)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1219,6 +1220,8 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Query: %s \n",query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1249,6 +1252,9 @@ PQsendQuery(PGconn *conn, const char *query)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery end :");
+
 	return 1;
 }
 
@@ -1266,6 +1272,8 @@ PQsendQueryParams(PGconn *conn,
 				  const int *paramFormats,
 				  int resultFormat)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1283,6 +1291,8 @@ PQsendQueryParams(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams end :");
+
 	return PQsendQueryGuts(conn,
 						   command,
 						   "",	/* use unnamed statement */
@@ -1306,6 +1316,8 @@ PQsendPrepare(PGconn *conn,
 			  const char *stmtName, const char *query,
 			  int nParams, const Oid *paramTypes)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1337,6 +1349,8 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Query: %s \n",stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1386,6 +1400,9 @@ PQsendPrepare(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare end :");
+
 	return 1;
 
 sendFailed:
@@ -1492,6 +1509,8 @@ PQsendQueryGuts(PGconn *conn,
 {
 	int			i;
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts start :");
+
 	/* This isn't gonna work on a 2.0 server */
 	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 	{
@@ -1507,6 +1526,8 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Command: %s \n",stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1638,6 +1659,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts end :");
+
 	return 1;
 
 sendFailed:
@@ -1780,6 +1804,8 @@ PQgetResult(PGconn *conn)
 {
 	PGresult   *res;
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult start :");
+
 	if (!conn)
 		return NULL;
 
@@ -1820,6 +1846,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -1874,6 +1901,8 @@ PQgetResult(PGconn *conn)
 		}
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult end :");
+
 	return res;
 }
 
@@ -2203,6 +2232,8 @@ PQsendDescribePortal(PGconn *conn, const char *portal)
 static int
 PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe start :");
+
 	/* Treat null desc_target as empty string */
 	if (!desc_target)
 		desc_target = "";
@@ -2218,6 +2249,8 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Describe type: %c, target: %s \n",desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
@@ -2249,6 +2282,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe end :");
+
 	return 1;
 
 sendFailed:
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 46ece1a..4c77749 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,186 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn,char* filename);
+static void traceLog_fputnbytes(PGconn *conn, int loglevel ,const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime,int type);
+#define	TRACELOG_TIME_SIZE	28
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime,int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond);
+	else if(type==1)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond,
+				localTime.wMilliseconds);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm);
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn,char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime,0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#endif
+}
+
+/* 
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn,logfilename);
+	conn->traceDebug=fopen(logfilename,"w");
+	fprintf(conn->traceDebug, "Max log size is %sB, log min level is %s\n",
+						 conn->logsize_str, conn->logminlevel_str);
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, int loglevel, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret;
+	char		currenttime[TRACELOG_TIME_SIZE];
+	bool		output_tracelog = true;
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if(conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+	else if(conn->traceDebug && output_tracelog)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+			fprintf(conn->traceDebug, "Max log size is %sB, log min level is %s\n",
+								 conn->logsize_str, conn->logminlevel_str);
+		}
+
+		/* Select trace log message style */
+		if(loglevel == LEVEL1)
+		{
+			getCurrentTime(currenttime,1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", currenttime, msgBuf);
+		}
+
+		if(loglevel == LEVEL2)
+		{
+			ret = fprintf(conn->traceDebug, "%s",msgBuf);
+		}
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn,int loglevel, const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+	bool		output_tracelog = true;
+
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug,str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug && output_tracelog)
+	{
+		if(conn->logminlevel < loglevel)
+		{
+			return;
+		}
+
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug,str,n);
+		fprintf(conn->traceDebug,"\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +280,7 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +295,7 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	traceLog_fprintf(conn, LEVEL2, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +332,8 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	traceLog_fprintf(conn, LEVEL2, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +360,7 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	traceLog_fprintf(conn, LEVEL2, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +372,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,12 +381,8 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -223,15 +398,12 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 int
 pqSkipnchar(size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, conn->inBuffer + conn->inCursor, len);
 
 	conn->inCursor += len;
 
@@ -245,15 +417,12 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf,"To backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -292,8 +461,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +496,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	traceLog_fprintf(conn, LEVEL2, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +715,8 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +752,8 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +842,15 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
 retry3:
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend:");
+
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +938,14 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend:");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1022,8 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	traceLog_fprintf(conn, LEVEL1, "Start sending message to backend:");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1036,8 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	traceLog_fprintf(conn, LEVEL1, "End sending message to backend:");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
@@ -963,6 +1143,9 @@ pqFlush(PGconn *conn)
 	if (conn->Pfdebug)
 		fflush(conn->Pfdebug);
 
+	if (conn->traceDebug)
+		fflush(conn->traceDebug);
+
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 3f13ddf..dd8edf3 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -36,6 +36,10 @@ extern "C"
 #define PG_COPYRES_EVENTS		  0x04
 #define PG_COPYRES_NOTICEHOOKS	  0x08
 
+/* trace log level */
+#define LEVEL1	10		/* Time and process (by default) */
+#define LEVEL2	11		/* Server and Client exchange messages */
+
 /* Application-visible enum types */
 
 /*
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66fd317..4f9726a 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -500,6 +500,13 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logdir;			/* Trace log directory to save log */
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	char	   *logminlevel_str;		/* Trace log level (string)*/
+	int			logsize;		/* Trace log maximum size */
+	int                     logminlevel;              /* Trace log level( "level1"(default) or "level2") */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -666,6 +673,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, int loglevel, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#23Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#22)
1 attachment(s)
RE: libpq debug log

Hi,

I created v5 patch. Please find attached the patch.

This patch updated following items;
- Changed "else if" to "if" in traceLog_fprintf(). This means that both PQtrace() and new trace log are set, you can get both log result.
- Implemented loglevel with enum. This change makes it easier if you want to add new log levels.
- Checked http://commitfest.cputube.org/, I modified the code slightly.

I would appreciate if you could review my latest patch.

Regards,
Aya Iwata

Attachments:

v5-0001-libpq-trace-log.patchapplication/octet-stream; name=v5-0001-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d2e5b08..080d24c 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1589,6 +1589,42 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </para>
       </listitem>
     </varlistentry>
+
+     <varlistentry id="libpq-connect-log-dir" xreflabel="logdir">
+      <term><literal>logdir</literal></term>
+      <listitem>
+       <para>
+        Path of directory where log file written. When both <literal>logdir</literal>
+        and <literal>logsize</literal> is set, logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-size" xreflabel="logsize">
+      <term><literal>logsize</literal></term>
+      <listitem>
+       <para>
+        Maximum log size. The unit is megabyte. When the log file size exceeds 
+        to <literal>logsize</literal>, the log is output to another file. 
+        When both <literal>logdir</literal> and <literal>logsize</literal> is set, 
+        logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-min-level" xreflabel="logminlevel">
+      <term><literal>logminlevel</literal></term>
+      <listitem>
+       <para>
+        Level of the log. <literal>level1</literal> and <literal>level2</literal> 
+        can be set, <literal>level1</literal> is the default. 
+        When this parameter is <literal>level1</literal>,it outputs the time 
+        in the function and connection time. <literal>level2</literal> outputs 
+        information on the protocol being exchanged in addition to 
+        <literal>level1</literal> information.
+      </para>
+     </listitem>
+    </varlistentry>
     </variablelist>
    </para>
   </sect2>
@@ -7472,6 +7508,36 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
       linkend="libpq-connect-target-session-attrs"/> connection parameter.
      </para>
     </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGDIR</envar></primary>
+      </indexterm>
+      <envar>PGLOGDIR</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-dir"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGSIZE</envar></primary>
+      </indexterm>
+      <envar>PGLOGSIZE</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-size"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGMINLEVEL</envar></primary>
+      </indexterm>
+      <envar>PGLOGMINLEVEL</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-min-level"/> connection parameter.
+     </para>
+    </listitem>
    </itemizedlist>
   </para>
 
@@ -8300,6 +8366,38 @@ int PQisthreadsafe();
   </para>
  </sect1>
 
+ <sect1 id="libpq-logging">
+  <title>Logging</title>
+
+  <indexterm zone="libpq-logging">
+   <primary>trace log</primary>
+  </indexterm>
+  <indexterm zone="libpq-logging">
+   <primary>logging</primary>
+  </indexterm>
+
+  <para>
+   Libpq trace log can trace information about SQL queries which issued by 
+   the libpq application. This output help determine whether the couse is 
+   on backend side or application side and solve issues with the libpq Driver 
+   when it use. The log without requiring recompile of the libpq application.
+  </para>
+
+  <para>
+   You can activate this log by setting both <parameter>logdir</parameter> and 
+   <parameter>logsize</parameter> of the connection string, or by setting both 
+   the environment variables <envar>PGLOGDIR</envar> and <envar>PGLOGSIZE</envar>.
+   The log trace file is written in a directory setting by <envar>PGLOGDIR</envar> 
+   or <parameter>logdir</parameter>.
+   Information to be output to the log file can be controlled by setting 
+   <parameter>logminlevel</parameter> or <envar>PGLOGMINLEVEL</envar>.
+   The file name is determined as <filename>libpq-%ProcessID-%Y-%m-%d_%H%M%S.log</filename>.
+   If the setting of the file path by the connection string or the environment variable is 
+   incorrect, the log file is not created in the intended location. 
+   The maximum log file size and log level you set is output to the beginning of the file, 
+   so you can check it.
+  </para>
+ </sect1>
 
  <sect1 id="libpq-build">
   <title>Building <application>libpq</application> Programs</title>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bc456fe..0080209 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
 #else
 #define DefaultSSLMode	"disable"
 #endif
+#define DefaultLogMinLevel	"LEVEL1"
 
 /* ----------
  * Definition of the conninfo parameters and their fallback resources.
@@ -325,6 +326,19 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
+	{"logminlevel", "PGLOGMINLEVEL", DefaultLogMinLevel, NULL,
+		"LogMinlevel", "", 7,
+	offsetof(struct pg_conn, logminlevel_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1142,26 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+
+		if(strcmp(conn->logminlevel_str, "level1") == 0)
+			conn->logminlevel = LEVEL1;
+
+		if(strcmp(conn->logminlevel_str, "level2") == 0)
+			conn->logminlevel = LEVEL2;
+
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3716,6 +3750,16 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->logminlevel_str)
+		free(conn->logminlevel_str);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3751,6 +3795,7 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Send connection terminate message to backend: ");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4153,6 +4198,8 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	traceLog_fprintf(conn, LEVEL1, "Send connection start message to backend: ");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 6aed8c8..42279a6 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,8 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	traceLog_fprintf(conn, LEVEL2, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1208,6 +1207,8 @@ fail:
 int
 PQsendQuery(PGconn *conn, const char *query)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1219,6 +1220,8 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Query: %s \n",query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1249,6 +1252,9 @@ PQsendQuery(PGconn *conn, const char *query)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery end :");
+
 	return 1;
 }
 
@@ -1266,6 +1272,8 @@ PQsendQueryParams(PGconn *conn,
 				  const int *paramFormats,
 				  int resultFormat)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1283,6 +1291,8 @@ PQsendQueryParams(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams end :");
+
 	return PQsendQueryGuts(conn,
 						   command,
 						   "",	/* use unnamed statement */
@@ -1306,6 +1316,8 @@ PQsendPrepare(PGconn *conn,
 			  const char *stmtName, const char *query,
 			  int nParams, const Oid *paramTypes)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare start :");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1337,6 +1349,8 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Query: %s \n",stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1386,6 +1400,9 @@ PQsendPrepare(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare end :");
+
 	return 1;
 
 sendFailed:
@@ -1492,6 +1509,8 @@ PQsendQueryGuts(PGconn *conn,
 {
 	int			i;
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts start :");
+
 	/* This isn't gonna work on a 2.0 server */
 	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 	{
@@ -1507,6 +1526,8 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Command: %s \n",stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1638,6 +1659,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts end :");
+
 	return 1;
 
 sendFailed:
@@ -1780,6 +1804,8 @@ PQgetResult(PGconn *conn)
 {
 	PGresult   *res;
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult start :");
+
 	if (!conn)
 		return NULL;
 
@@ -1820,6 +1846,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -1874,6 +1901,8 @@ PQgetResult(PGconn *conn)
 		}
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult end :");
+
 	return res;
 }
 
@@ -2203,6 +2232,8 @@ PQsendDescribePortal(PGconn *conn, const char *portal)
 static int
 PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe start :");
+
 	/* Treat null desc_target as empty string */
 	if (!desc_target)
 		desc_target = "";
@@ -2218,6 +2249,8 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Describe type: %c, target: %s \n",desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
@@ -2249,6 +2282,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe end :");
+
 	return 1;
 
 sendFailed:
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 46ece1a..5a4b6c3 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,191 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn,char* filename);
+static void traceLog_fputnbytes(PGconn *conn, int loglevel ,const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime,int type);
+#define	TRACELOG_TIME_SIZE	28
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime,int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond);
+	else if(type==1)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond,
+				localTime.wMilliseconds);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d/%02d/%02d %02d:%02d:%02d.%03d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm);
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn,char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime,0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#endif
+}
+
+/*
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn,logfilename);
+	conn->traceDebug=fopen(logfilename,"w");
+	fprintf(conn->traceDebug, "Max log size is %sB, log min level is %s\n",
+						 conn->logsize_str, conn->logminlevel_str);
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, PGTraceLogLevel loglevel, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret = 0;
+	char		currenttime[TRACELOG_TIME_SIZE];
+	bool		output_tracelog = true;
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	/* Determin whether to output the log message to the file */
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if(conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+
+	if(conn->traceDebug && output_tracelog)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+			fprintf(conn->traceDebug, "Max log size is %sB, log min level is %s\n",
+								 conn->logsize_str, conn->logminlevel_str);
+		}
+
+		/*
+		 * Select trace log message style. The output style of LEVEL1 or LEVEL2 is
+		 * always selected.
+		 */
+		if(loglevel == LEVEL1)
+		{
+			getCurrentTime(currenttime,1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", currenttime, msgBuf);
+		}
+
+		if(loglevel == LEVEL2)
+		{
+			ret = fprintf(conn->traceDebug, "%s",msgBuf);
+		}
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn,int loglevel, const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+	bool		output_tracelog = true;
+
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug,str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug && output_tracelog)
+	{
+		if(conn->logminlevel < loglevel)
+		{
+			return;
+		}
+
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug,str,n);
+		fprintf(conn->traceDebug,"\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +285,7 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +300,7 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	traceLog_fprintf(conn, LEVEL2, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +337,8 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	traceLog_fprintf(conn, LEVEL2, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +365,7 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	traceLog_fprintf(conn, LEVEL2, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +377,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,12 +386,8 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -223,15 +403,12 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 int
 pqSkipnchar(size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, conn->inBuffer + conn->inCursor, len);
 
 	conn->inCursor += len;
 
@@ -245,15 +422,12 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf,"To backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -292,8 +466,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +501,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	traceLog_fprintf(conn, LEVEL2, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +720,8 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +757,8 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +847,15 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
 retry3:
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend:");
+
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +943,14 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend:");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend:");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1027,8 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	traceLog_fprintf(conn, LEVEL1, "Start sending message to backend:");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1041,8 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	traceLog_fprintf(conn, LEVEL1, "End sending message to backend:");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
@@ -963,6 +1148,9 @@ pqFlush(PGconn *conn)
 	if (conn->Pfdebug)
 		fflush(conn->Pfdebug);
 
+	if (conn->traceDebug)
+		fflush(conn->traceDebug);
+
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 3f13ddf..aa3d56a 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -134,6 +134,16 @@ typedef enum
 	PQPING_NO_ATTEMPT			/* connection not attempted (bad params) */
 } PGPing;
 
+/*
+ * libpq trce log level
+ */
+
+typedef enum
+{
+	LEVEL1,				 /* Time and process (by default) */
+	LEVEL2				 /* Server and Client exchange messages */
+} PGTraceLogLevel;
+
 /* PGconn encapsulates a connection to the backend.
  * The contents of this struct are not supposed to be known to applications.
  */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 66fd317..1c8a749 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -500,6 +500,13 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logdir;			/* Trace log directory to save log */
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	char	   *logminlevel_str;		/* Trace log level (string)*/
+	int			logsize;		/* Trace log maximum size */
+	PGTraceLogLevel logminlevel;              /* Trace log level( "level1"(default) or "level2") */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -666,6 +673,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, PGTraceLogLevel loglevel, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#24Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#23)
RE: libpq debug log

Hi,

I have developed a new libpq trace logging aimed at checking which side (server or client) is causing the performance issue.

The new libpq trace log can do the following things;
- Setting whether to get log or not by using connection strings or environment variables. It means that application source code changes is not needed to get the log.
- Getting time when receive and send process start/end. Functions too.
- Setting log level; When level1(default) is set, it outputs the time in the function and connection time. When level2 is set, it outputs information on the protocol message being exchanged, in addition to level1 information.

I updated patch, but I am not sure if these changes and implementation are correct or not. So I need your comment and advice.
I would appreciate your advice and develop/fix my patch further.

Regards,
Aya Iwata

#25Nagaura, Ryohei
nagaura.ryohei@jp.fujitsu.com
In reply to: Iwata, Aya (#24)
RE: libpq debug log

Hi Iwata-san,

I used your patch for my private work, so I write my opinion and four feedback below.
On Fri, Jan 18, 2019 at 8:19 AM, Iwata, Aya wrote:

- Setting whether to get log or not by using connection strings or environment
variables. It means that application source code changes is not needed to get
the log.
- Getting time when receive and send process start/end. Functions too.

This merit was very helpful for my use, so I want your proposal function in postgres.

The followings are feedback from me.

1)
It would be better making the log format the same as the server log format, I think.
Your log format:
2019/01/22 04:15:25.496 ...
Server log format:
2019-01-22 04:15:25.496 UTC ...
There are two differences:
One is separator character of date, "/" and "-".
The another is standard time information.

2)
It was difficult for me to understand the first line message in the log file.
"Max log size is 10B, log min level is LEVEL1"
Does this mean as follows?
"The maximum size of this file is 10 Bytes, the parameter 'log min level' is set to LEVEL 1."

3)
Under the circumstance that the environment variables "PGLOGDIR" and "PGLOGSIZE" are set correctly,
the log file will also be created when the user connect the server with "psql".
Does this follow the specification you have thought?
Is there any option to unset only in that session when you want to connect with "psql"?

4)
Your patch affects the behavior of PQtrace().
The log of the existing PQtrace() is as follows:
From backend> "id"
From backend (#4)> 16387
From backend (#2)> 1
From backend (#4)> 23
...
Your patch makes PQtrace() including the following log in addition to the above.
To backend> Msg complete, length 27
Start sending message to backend:End sending message to backend:PQsendQuery end :PQgetResult start :Start receiving message from backend:End receiving message from backend:From backend> T
...

For your information.
Best regards,
---------------------
Ryohei Nagaura

#26Andres Freund
andres@anarazel.de
In reply to: Peter Eisentraut (#20)
Re: libpq debug log

Hi,

On 2018-11-28 23:20:03 +0100, Peter Eisentraut wrote:

This does not excite me. It seems mostly redundant with using tcpdump.

I think the one counter-argument to this is that using tcpdump in
real-world scenarios has become quite hard, due to encryption. Even with
access to the private key you cannot decrypt the stream. Wonder if the
solution to that would be an option to write out the decrypted data into
a .pcap or such.

Greetings,

Andres Freund

#27Jacob Champion
pchampion@pivotal.io
In reply to: Andres Freund (#26)
Re: libpq debug log

On Thu, Feb 14, 2019 at 10:17 AM Andres Freund <andres@anarazel.de> wrote:

On 2018-11-28 23:20:03 +0100, Peter Eisentraut wrote:

This does not excite me. It seems mostly redundant with using tcpdump.

I think the one counter-argument to this is that using tcpdump in
real-world scenarios has become quite hard, due to encryption.

+1. Another difficulty is having the OS permissions to do the raw
packet dumps in the first place.

--Jacob

#28Jamison, Kirk
k.jamison@jp.fujitsu.com
In reply to: Andres Freund (#26)
RE: libpq debug log

On February 14, 2019 6:16 PM +0000, Andres Freund wrote:

Hi,

On 2018-11-28 23:20:03 +0100, Peter Eisentraut wrote:

This does not excite me. It seems mostly redundant with using tcpdump.

I think the one counter-argument to this is that using tcpdump in real-world
scenarios has become quite hard, due to encryption. Even with access to the
private key you cannot decrypt the stream. Wonder if the solution to that
would be an option to write out the decrypted data into a .pcap or such.

I agree that network debug trace logs would be useful for users not knowledgeable
of postgres internals, so I understand the value of the feature,
as long as only necessary/digestible information is outputted.
I'll also check the patch later.

For Andres, I haven't looked into tcpdump yet, but I'd like to ask whether
or not the decrypted output to .pcap (if implemented) may be useful to
application users. What could be the limitations?
Could you explain a bit further on the idea?

Regards,
Kirk Jamison

#29Andres Freund
andres@anarazel.de
In reply to: Jamison, Kirk (#28)
1 attachment(s)
Re: libpq debug log

Hi,

On 2019-02-18 02:23:12 +0000, Jamison, Kirk wrote:

For Andres, I haven't looked into tcpdump yet, but I'd like to ask whether
or not the decrypted output to .pcap (if implemented) may be useful to
application users. What could be the limitations?
Could you explain a bit further on the idea?

Well, wireshark (and also tcpdump in a less comfortable manner) has a
dissector for the postgresql protocol. That allows to dig into various
parts. See e.g. the attached as an example of what you can see as the
response to a SELECT 1;

Right now that's not usable if the connection is via TLS, as pretty much
all encrypted connection use some form of forward secrecy, so even if
you had access the the private key, we'd not be able to parse it into an
unencrypted manner.

Greetings,

Andres Freund

Attachments:

wireshark.pngimage/pngDownload
�PNG


IHDR	��,2q�	pHYs	9�R
tIME�#/�(��tEXtCommentCreated with GIMPW� IDATx���wX�����F;���*J�P�wc	"�����f���D�1�M1�c5��{o(b���q�+�����5�#H9�h������������������p$	���.x�!���C����O�777����}��{��Z��_;��:u��{�������_n<�u�<o���V�����&������o�W�K�.�n�z��^����uj<�f^��[�����/_�����'���p&�K�.��M#�dff��s���\�dI�#�O�n���J�3fL�>}��p�����;::��������Mj���*?v��������m;f��7n4��9s���,�w�}gnn��Nw��6l����������\�B��}{���Z���������io���V����}��W�	���?���k��z�{�nB���]\\Z����+��'���3G�������<77k�:��]�jU���v�jdd4e���q6����@
�l�������k����w��iZ{d��]|>?,,����_��-[���93"""&&&!!a����.]***j�����n��}���7�|s����E-����s�����p�-���:22�O�>			����?>����*�G`�o�:�x<��hcc�c�����c_�'`HH��7�q[\\,��<x�}����>�lllj~�a�F�����s��Q:�~�|���8N}�����788���u��Q���z�
���
���c��u��������>���.:q�D�R����������{�!C�TUU]�z�}[QQq�����(���v�m�!f?
|��7�����w�����J��_~�d��i��9;;o������RUU5w��v��y{{/Y���)U�R��W_-^�����rpp�������	!G�


urr���������O�<quu]�bE��������[G��g��C�V�Z���;w�\B����v��E�/^L��y����g���a��	������+W����B�`��������%��X�������),,������?~���k�o{��y���%K��������E�!EEES�N���������_�z����%%%�g�655533���������-Zt��U__��������0��@�L6u�Tww�>}�TVV��??77w��	���[�n��^nAA���S������g��MIII<x���[�6m�~��&5G{'�|p��F�o���y���_}��i�{����gtt����[����]\\f���R���.\sqq8p`\\{���{��U�������R�m�V[���������P�f���B���G�~�zhhh��o������H���g��1���#Gv������*�����������|���d2Y�����?d�77����.��?�����}�?����]�5k����{xxL�<�������j��4������^�b���w�����+�J��h��*�t������o�=y�d��A�'Of���feeM�>��~HII���x6��u�V�P��];�k
�������;��tss�5���F��������d2Yll,�0�������W�����H$���z��~��7n�8{����'�\l�*/--��t-!���BLLL6l������_-_�\�tpyy9��9{�������]{���1c��9r���qqq
/�|��������������H��c��F�%N�8Q��L�<9::z������.{��������f�z����ry������5{	!�=���<x�_|������t��ccc��&M�dkk����[�?~�I���������;��;wN"���������=z������:���a�y�f�\������������\����a��mqqq'N��df�������o�NJJz��7	!�/<xpZZ��G��z���N��/]�4e���/���o���T*e��������w���w���}�!���'N����RSS###���M����8q����O�<��o��=z�������G����6���^�v�r������k��S~k9z�����z��=�+V��q���sqqq���K�.�Y���$<<|������?����i����	!o�����k��
�<�����]�{��6��@�������+��=~����W�����tJ���?������c������q���3����\�����������z6�������Q�!����g���y<����ojj���?~�����L���;�)��6������������]��������G���������v:*))
�fff����Hv��S�!{�������������k��&5���T(���?�Q�����kGQT���G�����*jii9v�X6	�#��={B8N�G�B�8p�@DD��9rd�5�	!������b�����666\.w��	����&4(+++99��k��Q�Fi'����M�8���������G�����d�GG��3g<x�I�-N�:emm�`�//�������&����|~QQQJJ
����������/_.�A�������������]���]���m�����u���]��g������kgggee5|������w}HH�����|��i��b�v����3�|>�����[8**j�������v�7n\S[��z�_���{w�[��k3f���;�7m����_���.\����9p����xx8��
�������	!<���R�T,w���9]�={���1������p����N��J�
��v�5�Lj>����+�����|>���5kV�A@�i/�O?��M�6�wiVV��7jf�
��i���2d��e���|>���������\y<v�����[7�#G�t��=66�]yU�F���^�z577744���&:::..��,Q,,,�JeYY������T*��!C
�L&+//wvvfK�������Q�D"y6�{���o��&%%�a�T:u�T��P(�D�kv����lmm[p$?~���A�B"##�ZXX�N����`�p�\>�_UU%�����n����|.�[PP�]��%���w������������g��4=m��~���?�=m�Z���Y]]�������q��������w�y��w�9t����_��'���O�4���j�����y}?/���qpp`W�����o������{[YY}��������k���...�O��Fkk��F;�����w�v����`hh����777�t��#�a��5�E!!!k���J�yyy���NNN��O�J�<�s��v�������Y�f��a�JUs�������o�?V �TUU�l��}���-���gy�<�T�n���
�yyy
�mK�������2���h_�$5G�H��/;���2������3g�l���
dggw��m�����Z_��f�Y=n���;wfee�����
��kThh�������~�m��[�����K�������������'O&��9Z6/bdd$�����|gff&���|||���8Ps�V�a8��)S>�����H.�����R*��kzz:�V�i7�,--����.%�R#a��
�B���F���gO}��fdd|��G�����!�[�n������2eJ�^�x<^pp0{����������o��mbb�������4i���*����*
?����>��������~~~�F������cnn�R��93���~����\�t)22�w��MZY�;]�t�����}���>|���B���������g��vqq>|���aS[��kW�\�q�����cdd����q�FKKK�g�kG����������;v�X__���m���������{�*����W��=;&&�y\);;;�����B�Rioo�h�6�Lj>�������v����k��~�L����?��s�a����C�=;��Q�FQ!d���.\HOO'�8p��)�&KKK�J��<5���c�\��a��gW{n�Q�
����...����.]�����Op��B��_~��_l��1''�������eeelb)""b����L�~���s��B��������M�6������9r��> �TTTx{{s�����c����P��e��)����������[[[?y�D[���������������m�H(,,�p���m������>�9�����
CC��m�B��=����l�.]��	W�{<�{�������\N��������E�*++i�NNN��������o��&99Y�P���o���MbY[[gff�T*�	?���
�+W�<~��av��g/���;t���'�TVVVWW_�~�r�����"�cmmMQT��[�No��<y�u�V�F�o�����~���b�������O�V��6l�J��{�n�.>q����;�X���Y�t��v�Zv�lBHpp���ku��?e��O?�4''�����]��5j�����}���T*�Ry�����,�\~��!�\�.��^��q�"""�������\��_8P,7����Lt�+��5k���$���_�>�P�K����g���������KPP����k�������k())qqq����&L�0|�������;�������~��gOWW�����Fttt

����s������fff��566����&u7y���k����3  �m�����[�h���C	!K�,������o���g������x��_�u��m�;w������������?}���#G.^������&&&:t���=z��y���{'L�p��]77����B�-[�y��~��}���|#a��]���0`���f�����R���:u������gDD��+W�������JHH�f��n��&����������;��e������]������3��HXw&&&999c��qss4h����/��B	sss�����������������...���C�e����s�,Y����a�mI��u����R??�����3M�]���GWW���'�^�Z�Z������~��]�~���}���[�l��r�,gg�-[�,^��M�6;v���k���q�dkk��[7�Z��G�ZTT����{����:W{�e���!!!��
srr1bDbbb�Omll����y�f//��;~�����a�
6t�����}���� yW���_�>}����;v�(��W�^�K�>�3���XC����K���?��C��h�D"A/��'O���SsI�W���{���3g��Z��7o>u���m��S�s��577���/��//oo�]�v���=����S(�����I���������K��g5��577�u�9�~6����@��T�h�sx��k=g~����������c���g�Y��Z��;+7���
_���z�s}�@���3�K�B��xyyy��������R��4�^x�a�_��R�/=�0���)>>^*�����`�fff����0.��#����P|||UU���+E��

4Mgdd�D��;����<t@�$�J=<<8�Z�n�j�\���kjjjk���b�_��KKK���
�B�����z�������a�L&�J+**�j�J�R�T^^^b��I�(������������bB����������������K�'nX_YU�KI_0b�(G��fhA
�����|�k�g��l?���Q���WB�F���x
,����@s��^�
�����)))���������E�?������cbb�=88�E:B�V�����|CCC�HdllLQ��G����u�%��<xobbbmmmgg����0Lii�D"IKK���������O�����TVU��[�5��0J�TkH��q4������jI5�a���-�U�:>r������v���HLJ:s���+W���MMM���zs�xW<�j�r�������������{����r�S��;�bn\�h4z|����t����_��q����4M�:NQT���A!M�0|��fv����k���������{���v|����=[�����������f�����~��ajjj�������������|��_���C>��h%>>>���U(������F���WXXhkk(
+++

,--������t�9I$���h�3l��Z������yyy���YYY������/�(�2��Tk����.F}��ly���&�Vs8�j�kkdRm���/����_������h�0k��v�����.�j���QEEeLl�����������s8�����V�*|���S�99:�o�}����������2YC�7���s���7�X[��qnEE�;vn�#�{���9��566�u���j������x�x3���T��/*nM4M��w���+�����D��Q���~�-5_��.��?~����w^z�~�j5�����;vlB�r���{����J/7�A=L333G��KU<����I'`kk��e�k&}���JKK������***������JKK���u�-99����;v�����������KIIIII!�xzzzzz���:;;�5���['O�������W��feeyyy��u'�����),M������
�p_�6�-����"4E.�!L��aoG�.���r��r��C3�"�rM��|:v�R���F�GZXX��GKJ����h�i-^�MvN������s���TUU-^���qcG�>��Ek�����-x���r����HJG�744z~O�\v��~ss?_{{�������k;��������A�#f��r����������S��t����������i�
��^�u|I��g��p:���������2y�>�~\����J+���/�f������}jgk���x5B��>	B�Ju��ySS�1c��U�e�f�M,g���L� ���d777�@�c����k�9�=����9-�h4�=244

����q�F^^��S���|KKK��J$���o8���I�Rm��m���5�L�������(*<<|���������g������o�o���������W3FB�����J5�d:�~�����@d������)F�al��!�>�|B�y�B�P\5Mh�0*�Xz7����������{#����tl��
q����k��n]x��Q�7o=HHX����}8�����u�--,������/�r����t���[�a��kWRSu71]����O�����v����������mCC��:}9���{��j4�H���w�*��*�S%�Q�V���8|�`Hp�������K�������^��L��������������������5�������|<�Kg��g���������m���o�������_�F$2���`Fl�RlH�s�70Tn��M�UI�t������{��C��&���T*��;���t�\��#�2����!|�Ow��o����;�9�E�V?z�������#--->>^�Vk?�(���111����Y4MGGGw�����)++k����4��cG33�3g��x<�GQ����ccc.\�����C����!C�4�����QP��]"�)�Qf��1��8c#���&���6R*.��G9���b��B;q��y�*F�P*�p�-5O*�i�6���Mz�-�"i��y����)�:C����q���Y�[��B�
B�;O�%���%ee���Z[�	�H�V4<N6m�<q���G���h3��9�>555�0.�������7�v���'O���'�e``���y����z'����������}��G�:ph���C


u�����sf�+�#EQ���
������w'�i����/!� IDATq�l�k���������z�����R���
�E9u��G���������[��f_�������b��4{����x�7={�Nysb�x�)oN����h��_�I�
�_��������*��Am�*wn�M����lf��������4m��\.?{�l�N��zv�����Gl��a��/�������� ((���Y�R%&&:99���EGG?������@ (//www���p8�n���T*6���hd2�P(����G)((����:u*����o_zz���~�j����������6y��j5�a���=!�����������aSG���d�*�V�}�����!C.�Cq!B�P� ;#�0B1��0���O����f���{s��~�|�8�����n���E������^!�Iok��E��~!�V�%����DJf�vL8rUzR=��5�u�i�AB�w�.������:���������r��$Mk)(,8q�X��>!!�/dmx"�9�������z7U��x�u��z��>�Z��]{v������4mll����
�Q���&��6g���{��9�{p���u�f~A���3�e����Sgn���������`1�&��{��


����������
���g��1���q����Z*D-�?!DV�Qk��gSCm�^rC�=�����j
��+�	�_�Ry�����77�/~~|7�B��e�0��7G#�D"
k��k=�juBB������������cX,WUU��t�P�P<x����^�r���mc������G�f���������
����			�����o�4iRHH���������u1�Q��u�V������I��;!�_)>��Q�a���pEq45���(�F�����4��P�������bC����i��NKO�6k����67kC��H���~zz�����+�s�A�nb����44��h�lk��V4n��;cy\E1�iE����M���\���k}eD&&B�@�P���M}����C�t4^?#��!$�&N�T�����o~���s}� k��s�����zv��(N�*.��av���e1c�9889v�O���nmt?
�j��������������w;�T��%���KNNJH|����G�W^��M��>w��M�"S/�v-����i:zQ>\p��qw.���;v�?vL�G����B����.\�/iZg����u�`�\�&�(�57
��*7
�*
[�V���i�;<WMH���y�����G�%�<�/���}��a<\�
�X,n�
NOO777���������oMb�X�T��hmII������]VV����������e������������9�V�������(�JHH���E����o�����&&&%%%���Mj�RE�K�wWT�i�n�~��!�SYU���of&�r������������D"�P(�V�D"333����M�P����4���og��aZz�[3�|���v^m�\���G������vrt���e�f��B(�5�pTC3#{z-��0%������833������U*]���(��������w���;f\)7����)�Os����9���h�u~K.���a}�������5�����O�\���-jZI�������q��EF��w`O�T���Y�&����W���[��s��[�	Cn�}Qrss�D_�;�F��+�[��W����v��;vm311qpp��f�B}�J�����R(R�l�(�
��6�ib"273
	���-����N��B�B������6Z�g���2����?����J]�����������*5-�Q���:��������ZM��T���B������UVV��_��[2��������%q��t�@ ��6lR���R�L��S������RKKKCCC###CCC�D���t�\ccc��y<�:���������h4���
[��M�V�^���gO�~�����?�p����D�����9;;[ZZ;884�����"C�s�r5Equ�@���5w��KK��Z^^�����_�������}���������"B���5�u�:OCdfd2�����YY>���_Wedd/�fe�D�`o5&�����b�����e�f�i�yff�K�M}\=Jnf+(cme%�&&%uhW�D��'O�-�ur�\������8a���cO~����s���I_�M���/�����h���h����9��p1��R�*�����O�0i��]U�������B��u)�!��?
**+N�>>�������+�s��<^i��1���G��=Vd"j��8r����S��]yL�������6�|���Qn��.j'))���C��/�@���}!q�+*�"S�u��i��CI��)��%=O���Qs��Z��b ����B>��mI����\�W�i��\wx�tJ�T��w�<8�����
];�K��[6��+�6�&++���������������w���U��t�P+�H���!)))l_m��e��������/���s'N�x�������^�|999y��ab��������P\#c3�S�J������	��d2�H����B���j[PP����\�c��2Y���Oaa�LV�V��\=����������w��dfe�3w�Z�)�H�lm���G�)��E����VU?�V��o��G��l ��i��	���Qs~A~Jr���3Nd|w1������o�Fk��OWW������\�&���?���{���5>u[���'c������^��S�#����'��l�kc���.
!=|A�����'��~}��mRG	�|k'G�����(�������B\M}?<,)-),,�������l�f��mtM�o2D��b���i��C/�>^�����T$j��j�
d����1r&�4�{B�9T=������JS�@HPO�{��S����;���"���o����2�����4�T�44M���V����L[[[CC�3g�X[[����L���r�k��9��)���ooo�w�W�V���[YY9r���W�\y����.�i�����URRRS���8F�Sc��Z��8G��!�P�S�T�nn���
�j5M�4����u��>77O*-����p��M�%�!||�:��O�X���G����a�R�B���i��{3���M}�01��ncF������CG�G��}=JJ^�h�5_�~�G�����s�������[��i��ZN������G�k�����h�~Mc>Sn�������������7l������K�/XZXv������+����EEu��q�>
<���w����}�7��:w�z�������[�|"mhI���=U*U���aa�n���|��)���%���~�j���F�Pr�c{;��"D�e�C��'w���33��W�P�s������U�j���!�WN���������q��m=���)#�i��i�!�0M�C��4�T�h�a�jukk�R��H$���KJJ��������5�)�lmm?~L�4���oL??��+W6����y�n��a������r8"����w���5\G��4�����cE�L,������D"C??/�Tjff�T����j�����uF��R�
�N$
66�"Q�C��BH�.�ll��lEU�f�����O���-5�LE�V~;��ws���"#,-,����%[���v���_~
�z���hx<�(�]�Y;��E4Z�����a���)������OF��g3��MZ��5�����fG:m����W�q�c�kEVvf~~��IS�=Y���f}���<�d���s7��gVv�����!��\���O�<6����{"�Sg�6�.C����1;;�&��_��w��%r�h�������f�Z�}��E�,l�������f&|B�Z����MNN����r;���T���:����4���y8�Q�5�F�T���~

����\.;�������Y�)M�����0�����i�V�T�TnaaQRR����������P(�^�=��][�p�^�!���<��(OOOBHqq���E�����C#e,���:#�!��{���P�������:yUT�JK�MM
rr
��
����d�%������	�F��x�?����G����wr���tNn��Y�/������f�����t;�����#�l��+�
y����h������u����5u����"����*=3��Z%����		=w����3�8�u��5?�����4����g=���q2���������p��]����W���
#]����n3�7T���Z�Mm��;����������
�3�/<=������w��rrtn�E��{����o������k��T�Z�����~�}�NIiI���/}�/������lf�����K���ddl�f��wg���5qv������>�����QK+����o��w�{`�=�
hd��Z��������t���D�0�fhZ�n��4��N�dC�R��@�T��F�4]ZZ���[]]Mqww4M'&&v��A�PTUUi����_�.[XX���B<==��;���=q�����9s���1c��������4h!���?������r���!R����i���8���1���5����3a!
�"/�H&�*((�LSS3<=]�B~yy�R�������eeR''{1��j�OC~�����|K���J.W����Y��|�����������c1���
���Z����>�G��fN�[h�g����{o������U|������>x�p�����F~���M�������e��m����=����Y�t���ot+^m�������2�a��]�\�]�b`@p`@�J�����]�]
�D�}lo����[/�?w1���v�WO��s��B�Vf�dO���.�-�����y�2�:,58�f�d������<h����mlFL�]���1��B�8����+
U�7cF�G6i/������_H\BEQ+�-���{7o}c�8��h��y��'���5���	��E���Qk`��}����?��5k$������� 
��LR�4�&����OW�eB�L�������hU�,**233���O�������������*�������kgff&�HD"��	lKKK�Z___.�K�tVV��
F�5c�����s��UWW���S.����SEQ���/!����������(b(��/�l(���q�Bh�F-
�b1��
4M�r�@`�0���r�&&&UU�\.!DM�<q�i�R�i����Bg'�CME�Y�&m��33+g��S'�kN��C��8� �@��L�{Q5[�Rkv��H+���&m�W����D>�:i�*�
B�0�N����O��M��W�>�7ol��i��qvA��xGV-%�������{��/��r]FEQr�|����
{�fo�2Z�B��YAk
�7o����>���y&cc���(��J�:z�hIi���gN����6d���W��?SUUI���Y�n��������u���rMLDfb�aC��Hd"4`������N�l�t��O��������������LD�#�xg�L��v��oW����	�"����E��������/�qs{����������+*&O����H�7---88��Dy�fM�u�{�����vV��*���o-�	����i�F&�����G����W��������4MK$WWW�F��h��gaaQYY�������g�###;;;B������[������/wrr�J�E���;;;geeUVV�7�W��������(�g���cckk3��j��%7������@XY���x<����F���p02�a��/M�+d?��kAa������},��dC8�������~�(+;g��}aa������/��u��k{.�?L���lY.���%�9�5����BM�����&k��&-=C�V+���+��m�FN6`~*0 H,o��e��Q_�?�:zfZ�}B���n��_��)������h��~]����FS��W�H�7�A�"�w+�wz�&/e�������g;��El*NJJ������ncc3����fu-��}����}-M��3�b�4�T*gN���X��=Bz������S�+3O=��2���0�I��n�������u����:Z�����;��)���������s���G�VTV��v���;~�>k�����B����v�$���7}�C`�g`5���\.W(������i4����5�{XL177/..fs��[��"�T���j���$�A�TJ������-((��d��������DRRRB122���455m�~�P��c�����5***������j�z���]�t���ommm�l�bddt��Y�SRR���������V�cbb����BaS��%����&�[�K�y�_����Z(�h4*s##"��
�@@�����G��&M���������\g'����X[��I�!����7u������?Y�m�Z�BB����0+��KH�$�I~��X[�����I]�:;4�zon=s�[/_V���;yr�`}���lgld|������E����]|��6�����sf���3����O7o��!t6����{���wn���3��`{{�&�I��C�|E�5\=V�~�'�MZd���m=���=�vM��:��uB�����[:������!^�w��'�T��U
�\�Q�i>�2r�
�"C^HG�q}��ZxY�=w?��A3�B�����t������q�O�='�Hh�f����7k�%
��������
�����uvv�p8NgFS_������D�Pj������o���t)�JOO�LMM���022
+..V��666E���G���r�Zmbb�K��������}������/\���o�MOO�v���+Wx<^AA�_|���SSS���)�rww_�p�@ �z�*!�c���%�p�KQ��S!�����V��)}�Z\ffv`�wU�:--�_�n����aa~��O�<��������3Y�!�6���:�_!�'&��
QT(�3���S�Vz�`[�,��m~���������v���t�h[��Y^n�o+����:*r���{�J�B��e��>�z�Q���S��cN�9���>���]>����L����LD9g���*������2I��c�X�����/��������K7�B���u�{�&�������G�.��G��
���5�(U�JSVYM�2�O�W�������kWO��������n���C8�&N@�Oi$������Q���.[w����js7��������������������J{P�V��q#((�������=X]]}��-�Xii���ivv�.!(�


:{���������U��o���~�F���i�����(���
���������������������n-4��f�~iB���,CG&S��{x8WV���qMNNU*�m=�
�jU�6�yy�2Y�Z]���B7�tB8K������T��������������CW/x+����}����_�~�s�����hia3Z��7�G8|��[Z�R����w������D4jD��+7��gHH���|��������y��������mm�F�o�L���>�����6m�^��&��?���^��^�yH'�Y�Q�kO��p�����������	����	v=?�N���^�v��9�h����n<p����!D�|���J���������VB��to�%��Eg/��M���IM[[���G7��mmm�j�.%�r���iiii��%%%�O������EI�����L�~�����C����at�{��������������4iR��}������SRR!���^^^������j���������������]9
���P4�?�[��4�n�M�!�
�j���B�����WJ�U�&WZ,�������ER���ZDC��0ML��!�"
?x���/�+�J�����N����_B�����_Vv��{wTj����=���z��������0g'��il������3�i���{�N������[�U������
/�L�$�} IDAT��P1o]l�B���r8������@w;yf�ey�(nf���ill��]=}��eR)M���162���h����j�w�_�J��h����i�[�&w����/�� ��>�occ��������(
KK�&��Z�V(��
5
M�"�������������������~���MLL$�����m�ZYY������;;;;;;6�V�������a���off���um�I��Lq����*%�0���K�|���M��61166�[Zt�*��P����X�'p�\!t}��U
���2����e��'��l=2�����9:9:;9:���5L(�>rt�5���F�(J�T��Z��3��(=�r���IoL���ghQQ��[��:7���Dk�����7�U*���q)�1���AvF:?X��u��S\k��kV/������/>��������j�h�'����-++cWB=�u������M���]R������y2������������V����>1���l���			'O�411�����l7I�R�DRYY���������h;;����g�[�0A.��j������X�B����Q�p�
�B"$���L
	���B	���
!B�<�i����!��@M����?�����~����5~�������u��t�OP��sW=*�������
�����
��h1]�F�
y��(�{�$-���q����d���z������/�lmm6������V��9��KV�qB��sg�Lxp�Ri����^�~���5�2�0B����^{���J�iii�����u�����R(��������]k��������Xl��	����[6V���������'�wB@s����^������t\���O��7o8�U����7�/R��;v�h��b1M�\.W�u����ri����=�_T\h��u233KOOwss��Z�YM�tzz���Yk���)++C/�NIIIR��eW���fff^^^�0.�
�_����x�Q���/&&��R�TWW�^j<���/5�����1�^x�a�/�K�W�MEE��c��/����@GGGooo�@��x�82�L�f���������kR,X�b��?%&&���sNNN�������~=zT3�����|��f�!�0�0�`a8�_t@+��[q�^~�RS�8��#�0���k������!����EM��`�\�����~sK���i��
�hhFC��"z
��8s�v��a���N�<E��C����G�)�����AO��xg�����0������u��a�=z�x���G��=����7q�D��K����w����������=��W�z�JmxYT)�?���L_���-#�R�}�R�k�3������~v��v�6o����3o�<>��:[���c``�/���urr�>}:!$--��X�jUDD���Sk�"�Lv�����$�\nf���w\S���a�
���l"nq��m�\��V}����j[W����q��SAP���lI���j^
�B���~���]9���p��quu�����������i��Q�

Q�N�A�1�,5x��19Ee��A_��x��`��Y���_�p�[����l{{����Z=z4!$""B��3�R
g��4b_ZZJ�k���t�����T������RkkkE@$��l��P >��k�.MM��c������{7//����!<W~��U]]���aee�����B�����u0�$����������b���Zq���/���i��-��+..~[i5f�--�����3=�+�����D333kk�������u�����<y���-,,����\nqq���[�������|������������omm��sg�����{wggg�/_����*((�p8����r��g����;w������C�Q��3��?~�xFFEQ����&M��x��V|�H$���233�B�����A�ttt�������p�B�C��K�f��%>+22��������������u�V��O�:UVVv��.�������w�����B--�~���o��NI??������s�*�(������=x� 44������{|>��qt�
@QT�@ZYYm���^�������N�������III7o����������C[�B�����@k+C5���8�a��-�UO�����������;����������)�
��AD�����q�F�-��j����������^�Z�OoV8e]��Y�MHH�������u�Vyy���&�=++k���,+"""::�G������5}������[�������B<<<�]�F��z�����^l{��aiiYUU�������]�J�������;���::::77�^hccc���,X��p�����[i������a�X,����O�<9~�xSSSMM�'O�888B���$?($$�����������tNMM��\UU�g����`gg������9�N�g��M�:U�����LLL������nnn��O��c322�U�U/����555uuu�f��(*))I����������������}������������2e��'pRir�W�������3����F300��?x������!\.w��������p8����d����--��/N�6-&&��������O?�p�Byyy�N���]��������G���{7,,,<<�����W���������a���Mee��Y�._�,�lllN�>����r��;vTUUn���S�N�g�����|��a��-[����w���o����B���������JJJ���6o�����3fL�o8���L�)J(��"J��q������b���D�.???��f�;u�$9FH��USSsss�7vhoo_ZZ���BHbb���K��HCCC�����������z�INN������e����u��x
�Yee���/	!fff��K-}���������R�=����N����&--���os=x���������f[ZZ��������:u���r[�fryy����7l����?`��y�����S<
���J<f�$z��!<� ++K�����Ypp���;w�����n��c��	�B��r�'s�(J��u��"�LPw7�//QRR��}���[�����������z�j������722�w����={�������k���7^�|y���qqqaaaW�\Y�l�������0q�D��v��]SS�������~�z%%������w��u+77��������8r����>��������������s���{��edd���?�K1`Z�Y$�g�s||������!�c��			�i��F�Eyy��p���X

���|��q8WW�����={&%%
>���UTT�;w.;;[(
]]�z����7��lmm�zxzz���<x���{xx����~�n��}�2���/�������X,>�O�+wss�r�Jmm��G�,,,Z�JxiiiNN�x�;��755���-QWW���MMMCC�z�jjj�b-
��k8g766�����KK�)S�<}������Dp8###��������;����z&'E(B�V���r��'�����^�J�G����/������po�=�:}�4����-**��f���}����k��A�������5g��9���K�B<==�w�N�}����pwwwB��Y���]�����r?~��cG�9����:11�[�n��������Kc8p <<�^r��~�����W������p:AAA���c��Ul~1$������|_�d_Y����
�@ x���P(\�f
!D(VWW����c�%%%�XzII�xF�@ �����2%��������?��]:K$]�pAYYy�����}LLL�455����o+++�'���g��={���������EZhi�������~����zyy��o
444���=z������)GJ��jii����=��0����}��g��?�����qc�v�����Z[[����>}�HN��(��b���F������%~���&�Rwuu���JJJ:v�8a�ccc4���("����SB�o)�]!���n��:�(�N���b�#��0��������W�\I��*���>|������_I,��kll���'~-> +++&&f����-�|�I~~��	������b�
''�~���o�MMM�����U��W&�����k�N|q����K���UUUm�<�w[��B�O�����S5k����������g�������{o��UYYY]]}��uWW�7�e]�tI ��������E��9����p����F___II�������
�������QRRBy��a���O�>-**�(JUU������hh�������N�7����~�����<'''9�C]]����~���������DOh~����W��O�q�FNN�:::����XYY���,X�������{k���G�}||������������_�|y������z��QZZ����:11��_~y��I�n�,X��_?����x�fzN+E��D%�^�~=L��������N��I+I���������ahhHo|����]�F��x�b�Y���=�/�+9Phnn>���7���;w�����/_���x����G��+��?���+<���Z�j�d�LLL�����������fff�u1��05���^�r?����%$$�����������EEE������c��;wVUU9::��x<##�_���bu��]�8XI���.\5jT�]AAA��KNN��x���O�<�w�������8���jbb�0�^�zUYY���LO{��7Z���gFF�����!>����x��	���E�n�N�>}���^�z���N�0���s�N�b�Xfffd>=..NUU���KW�z����������QRRB~+++O�<���K���������quu���d ��o�d�^�|��p��n���}���jjjh�Br���3}��@�"���}B�=�W!#��c�E	���j�>�?m�����j���AAA����i�����r��W�1b��*��S�N�5k������gEE����mff��}{]]]eee.����RRR�����������r���(j���+V���������_?---���HQTEE���+.\���WVV��O?5|�����+?7���M/���DctWKBHie���
����Wo����x�������������m2d��[mmm}}�������������1�������6�?������������������$c������r��������@|YB��������SS�I�&��������9s��?���F_3�|X�����A�
T��z�\�dI�$�$9��b``��@�4T��U�A�����A_u��aDDDDD�����O��CC�3f�X�m������g���#DG���o������/\�������~����s�>}�TKK+   $$$33s���/^����3f������/^�������v����_J^d��999C������������_YY�a���S�������7�����_cu�����
��"�|��f��t�u[����s�����{Q�RRR�B�,sa?���H��3�������*�|���u��.�Z)x?���O?�To���/~���,^{���[�Y�fI<i��z&�>|�p�-&L�0a��OO��������7�/X,��y����W�����k��Lcc���2����y��eajj�������}||n��-��:g�������y����fdd��������\�w�_����?l�0��&F��"Z�������[��[�nyyy5k]bY(��Vii��Mbb���W�-[����/����(=|�n��u���������������;v���m�
����?~\r
3@����lll8�-�z��R���F��0����R�Tx�a�/�{�{��m��{��b��
�5.!�W�^U�?~�����G��
�� ^Hy��8s�LQQQ+]\___UU�g��"��BQEQ��HD��wB������w�x�Wr��(6�M�Y������' ��Pl/�<�����%S�)(6�b�x!�>l$AC���?��3����o��A���,����t##���~VVV��]���7l�� ����?~��]������������b``@9r���U������%='I hhh4��m��1c�hiia�R+�<����{K�,����r��~�m`` ������_bb"�����tpp5j���3��f<�����nnn����[9*�B.m\l!���={����_���e���i�/^�� ��������s'N��������)����7w��)::�||������m83���_ee���W���r_�addt��a���q�===QtZ��		���Wzzzrr��e��5z��@ @�~$�l�R^^7��M�6�?�%W���8p`����@k����������/[���;"++���� ���H�|�������z����"��;w��������7��.������S��.�K�f�����E���s����f�200pssKHH��y���������������+V���YXX���WTTP��������9s�hii�������K�.���~�������l���/_��(*&&���Y|n�����=K���A��N�����e��������;88QUXX8a�;;�_~�)��#��X,�cff6l����8q������:%%%((������t���E������������?z�h��}��(���'���k�����wrr�����+DFF������o����(�"��J[[����[�n���S�L������RllllF����(������hll���AQ�����=�f�{{�Y�f5zm��}^^^���nnn�������R��8���ill<`�����(����k�:���SEE����a�	!;w������������n��M�������~�������N�<�b�
������5������v���uuu(��^x+���F���3EQ���#F�X�r��/�YYY�G���{w�>}v�����uu��t<\Z6Hns���;v�[���o�	

<xprr�����O�>p�@ee�~����[��]�������?�i�-Iy����I�&�������>|x���QQQ���s���>}:=�N�J��t���'�m�FQ���������=kaaA�#MQ���c===?~\TT4d���������u�������v��!::������^�|��.\�PWWG�EDDxxx����W���_�������7o���{xx<���Z]]}�������������v������/^$����7""����A�bSYYy��Y;;;z���W/_����,
CCC
t������:�}���G��x������E�!����u���/��"""�S�NYYY�@����B�!�����U�C IDATK�,9r�H�����q����q��b5Z������7n��������������������������r�����������?s�Lxx��G�~�������={N�6�2o��g�����WVV��������7%@,$$d��!"��BQ�H$���������]
��/�����o��yB���{���.]�����2u����7�?�=Bc��B���
��E"�H$��www:t(!$44��_]�|���R��})�z���������#""�'G.Y�$$$��D�cNy6�}���_�u����?\�~}�v������CB�w�}gggWTT$
)��EQ�u�B���G�n��]�v����033�Dnnn����������(.�kbb2c��C����y�^WX�n�����K�k.�������s��H6zt�Ho�p83g�;v���������;}���[HHHtt������04�aaa�\PPPll������mYl-Z�|����R��������p�BaBBBVV��_~�f�}}}CCC������&�!����s���3}}}E"���9!$..����Ru��m[XX���+EQ���k��IKK���m�:th���������>|���d8������VVVvuu�����EsC(
���|||APP�������='�DDDDFFjhhhhh,Z���_~�={6
3@����K���;v���+���G�����k����0KKK���/�&_7�Ho166�����jjj����o���***������f����p�S�|�P(d��([�)onn���?B������3s����(��fdd���rrr��_�����^]]]RRbmm-yXff�@ ��������G#/��b���������H�������044\�`�����'JbMZuuu%%%z���
�x����������4zZ��i��hx��ROOO|����|�����#F���(++��L���������K�G����_i�dgg:�^�l�j�T233cbb���'����occ�h	���377�7����.N��a��i!�(J6����TQQ���%�������2q�����_C�����-�����
�l�/_���ejj��JSo���1***���{��qqq�w5�-S���x��I����KZff&}Laaamm���iNNNmm��������tFddd��lnn���#9��������G����;w�����h^���n������W������m�1�4i��e�F���p�RJQ���Hk*6)(�m\l���

�5B���I^^��WRR"�<���8�%���c����G��?K��-�z��@7z�F�
o��z%���$;;�����-�%ki�B���^/<�Weff��$N4f�fi|r�����I�6n�H�<x0�U�@�c��W�^����r���-[�|�)8e���K����B�����?�R����������555��=��c���!d�����mKKK���^�bE�~����mll***���	!'O����l���|�������l�H_QQ���doo��W_UTT�D�����w������������?~��|}}��udd��/X,���!=�P__���.77W��+//wvv�p8���'O��7J�����q...�:|>���;�


%�fdd03i��-[���s�~�sFF��#[C]]]�uuuS�LY�~}ll,EQ�������x�g�����m��������`�I��-�9��,--���[�nEh�1��q�g�����?��?���ZXX���g����Mspp���PSS��B=~���3���h8���.��
�=5�����_�[�.88����fff'N���7�s����ggg���hkk��������(j��a����������a=�z���&L044tss���#il����K�U�KKK<�����_-_����������n����F����9s������agg7m��3f�{��7n,^�������h������������>}���.��k���6m������]��2��E�5���)����,6��=������{��E:t022Z�f���EQ����<y���U�>}�m���1�z����o������cff�~�z�F�ly�7���y����G�

��u�����g�={�LSS3  `����J`hhhjjj�=x<���9B/R-��K�J�IhV�3����o�]�t�����=ztXX�	!�������
gn��g��3g������������� ��Pl/B����
�����h�������V����������
�a"����=z���Y�6<h)(6�b�x	�K�.����������8p`���
�JTUU��D"�	���<����G/�H�j����0%%�������c��}�%�����'�������2RPl��jH��jZ���
�v���9S^^��h�A�t�{�O����e���������v\~Y������w'0l�����$xO-Z���������~��vV��^�t����O2�6������UUx�H����Wg?�J��@ ��������{w�tC�4�����������K�45�}\��tf�XR�K��W\Z��������]���~��wB��_�����5-t�,���t�NL����r5oW���]��K;������J^����/)9�z����E�G��.�1�%�����(6(6m,�~��s�n��{P'BH|r���1��%j�����wUV�B�����_@a�X:Z�����h�A�����?o�-�����I��,}J��4�5�W�FcZXTr����Y9<e��vC{�P�r��,_�,�	C�=��	!l���\j�u��F?HK/+�P��\��������x���/����EYVQ�r���w����n����E+fO�PS#�\�{;>����Z������p�Wg��=�Ing�z�L�C�����L����'���� ����sB����?�D�i��Z���f���j�n�Q+~�#���^��v7�l����o����D!���~1TX�Ht������++�44�\�t����M�Ji�A�%3>5��k�/5�,Q����k7�ji~5{:�8��s�Yv.�W�A������s�N����2]m�����]�Z�=��x)������B�KL���q�l�],B454l��

���b�e����Q
o`�\����B���]Q-��"�Pn���!C�e��{�*�;����I!+e��[�����������a��Mzt���1����3��/���i%%���RR#/\7����ifn�����j������p��w���Y9'�D�VT��m2o�\�7vP?S#��E{��f�X���3�UUScef>v���oN+eC�rp���"%>r��cVf�M~��sY�x�
�M��������R��h���SG552 ����B�R��u�D������J��������|��A����N{t	EE����<9���C��ReN^�^US��Sf��,}J��0�_����a�kj7�;��h��Wye��Sg/���?�s����e�b�������3���
&	6��-xY�����S�>>��������.\�<b}��W
�t��x�o)��v7.8���N�Cks���������3W���[w?�;q�xL��)�9���R�5JKK����j�nTc}��We�6����k�����BB(_�b���n����pZ�PSC�����a�X��uf����KT
���<��U��%/�e�������1�����ofl�����g�4���,����d��F1`���rq�&���;��HQ����Wv�\8u"s.+�`��j�V��+��PD�!��C9d���`�����;YYEQM�n�JY���_�/+���������a��(.������H���ts�{������������H���8h�gf_���O*g)|��-��UU��zedeWV��oTp��6t"8�Z��h����
u5��RdC�rP�{�&>,)-�>jX����\�/^�b�b��x��<ee��B��������o�����'�bab���v�n,�N@��QVR���$���hB�<Z\Z����7Ye�e�>|�������:M�)�^���jV�����vB���]l�uuO�$���e��e��gfw�t��0#���3�'�E!�EH��>�v�������M�������?/�9����f\B�.~�J���-|�W����������,z����GKJ}r����#C��[i�!N�[q���e���>�=�}��O_H|��������UV��	>>��U���7�����3���;+�%!�fl�����%%jj>��}��������Y�'���d��~%�!���1��+*-����U���.!$�����{����0�u��*OEMU%%�Yye�m;�1�*+)1��|����f��;�X��������*37��Z)7i%J����D9�Z�Mz������G���	)�MVi��8�b�t����:{�ed�D��=�+����-�(��
������Y�Z����o9p��U)�6C��\����>JZ�&�?�v���)��_'N�������:����@(<~�jR����}m�~]��+�/�"�Pn��!C�e���|�H�Y���1tR����������=J��X^Y��`�d7���%+��e�s��e3���v}�C��f���jk5���!Y��o&�Bl,�n�%�(�aI��X,�bw�Y����q8BHFV������Ao�D��2S2�������jj*��w:�S�BCM���s7o�V�z"-^2�`-������{)5���
�e��E�(6(6�~]]N~�����-�f�f�L���V^[�]�h�N���w��b�S�]GZ����]!��0�Yf{K���'�>Q�){:9�y��b��|��|!76�OHN�p�������g��y9yQ;��g/u���01����%�n�'u��RH�I|hkia������^�#:�T������O��������a`H���	�o��<"���,��p�?�86��wcS2���8�X_�V|���z,JGKs��)w��55Z���&!�vB��+�����0Q\��H�@( yn�@���c��z����C(_wc����<��������	C���-�K�<�!f�����?��S�k��(yx��{t���Y�m��{q�:�1�P���Pa��O]���_`jd���(�i��%ahm��P��H���(Og��#O
�����G���C��A��6�/�P�$1����u/��,}�-�[l��������<���\Vx _�:�Z�ji�Mzhn��Mk�@W[��K��7�$����0JW[�N����OY��0�B�;^ET�[��7B��!s�e���|��-�J�NJGKS��;v�2���=~����@(DV*$+��e�s��e3���vq?�^�u'��`Tp��������j���{"O��O[KS ������H�)\�Y,[��TUT�[p��������������������BQeUu?�AA����"/^������*i��1�����Pw�a���s��,��(6(6o]yeE���z�G��mma����Q%Q�U���[����m���}��Oa�N���Y�;�����/���}�<RZKJ�2���:�L5,������|A]���MFY�~Y��5���gWm��H_���������\��_@�����v����
���_�D�����k7�Cb-!�b���J�tv����~����MV������)q9�v-Cj\�������9!���8���Fl�svBJ������!$���V\"}
���69�F����P���wb��\\��B��M'
XU][/��;���41$�d�|����^IY9EQ�O)�$g<���,)+�Sf�Y����=mm_7B����m;����&C(w��Va;���(.Y�k/�������K�Z�1�(9R�����������������,�A��6�/��7c\�:��f��>J�����O��x[mT��������������k��I}6�v;Sc]�Z~��\Vx _��X,?w��{���8�P$���Qw��#dA�>]���Gwrs9t�BvA!�����b(����4��V�{s�`H��i�}T��������ZAQ���)\�BV�<+��e�s��e7�����!�R���}#6~��C=]���TU,L���T��?���O��R�.��������szf��WD"QgO7��$�c��WVQy�nlk?FQ��W���;�Y*���������@=�_�S��v@L����E"�P(t�k?���*}�2Yy�)���L��
}J��a�������z�_WWo���G��/��p"C)B,MM�����,��H���d7��~Y>z�Z�������=�z�$c��;yt}S�����^���d���kz:;��r#>9��E(9�i
���hOqno�b���y�:5Y��e���<L��8u��dH��%�n�>w���`�M?����\_G[���9��:i	ERXT��Wr��y�s���(�����[�O���t�Baiy���g�}=��Skj��5�����D*+qeY`V���Pa/����x�'c�M��
_��<��p�w�"K�l.�%_j��q���|�Jt�H0T�v���+8�[EU�������������7x!���x�q�pS��h�?��O�TWS�wEQ"������~����������]�G�~�$����aZzuMm'wW��Z~�����<����![����"���H�7BM��F+,C0�mZr�}%a��;)6��������	Y)c�,_c.���&{Xx[>��_�P�7�tV~���cL

���)+q�����.N6V�����\�
O���Lg�X
�%_0���k��i��Y�)q��/]�����(��WTV	E"N�����LN�������R��ex?+>�
������p��+%7�	���<��w��b��UU?�LlXe�"Q��s��{���>��|�F�/��a�w�~��~�������a^��9����x4c����F�`�����_n]m-?7?7������z���tu!�::
�������c��vl���;��B�����o����y�:5Y�m�G��8����#���2���PVR��������P��*]$��2<*�@OW����P���C1����3548v��'C�[��'G���hs9���E��/��v�����6�e 1�)M�C��!��~17)Wb�����������8��������Z90��f���%�����������jN��p��i��R�)2t�P(�Tx���K�����)�F���(��
U������z��~6���^���"7�e��r7�:����m�$>ts��Ix�j�AS�u�w�
�������Q��U�4�r�KZ�(��[Y�;��!so�\Sd����>�m�JJ�NJKCY�����_nnc.��l���s�W(�:UQY���q��*�?HY��dd�x�8�[��g[��J��SS[[RV��������,���#�!�n����C�W�l��DM����.v�]�l�]�����Zoa���LLI���T���jn~)�,x�+>�
4lE��LS��k��K�}>i���>!��S�G�>N
�Lfn~^���#O�o��:�`�����j��3���)O�s�]���LZ����]L��aE%��Ga^��9��f&R�Por
�Jt���������r_=������4	!�L�	!5�|��������O�s[R~*�k�?5������>���u���B[S����jk�q��������������[#5�����
��WV������!���E�J��"��-(�27#����/+��PWGr���g�/K^u���:~�����u��������4�0?y�J�PO71%������J{K��G��uu��g�B9�C�bcaF��nA(�s��V���D��-l4�����$���5YvY��e~�d<���{8�s8����i.l���Z��o������$ IDAT���
��N}����������r�\Vx�����G�]G�
��S2�����E�������M���-(ly��@Z}����-M���!��}i5���F�J�f}%��&;)dekd%C�,wc.��li=,�u�����1�s�>�4��bU��BX,��x!�����G���M��-S�>���p��!�$�?���+������N �(B�HD�D"��NJJ\6�%�,!���c���Z���(.�p#���}#��Cg/(q���=���7(�{+eC����g�y�.��^�����������]�?�����s���G������/�Y�����U���x�����1	b�|6i������4��0\���jV�.���I��0������{��/^
���Qt�������QnI��\����J�<5((@_G�UY�����:f�������;{�]�/9}D$���M���K�&���^��d%.�����f�������W�?L���+c���Yv�_�N,�<�y�L�����u���mG�E%��:v������=	!n�v���z:;��_�}W�l8�����J��
��>v����U+����'��4x�����>AG�]�67354`�|�������tz�������w��H �+aR�(�@@�p8��9�r�/�
���p�^\{�vf���E�7b<����M*-�g!DWK��b1�(�R������������"��U
uUc}��WoVVU�y��=�!�����6�V����niZ����3���m��=�1������8u��TW;r����N�7C�l6[[S#��KBHIY��[wUx��o��L�!�~�u����/iET�[i���C�
�PS��m��Dm�W��I)��"+e���R��g�����s����������)�����j��Fv������;����d��3:����?-g�p��w��k����.]��2-t�s[���PSQ��p���JKC������]���a}�T���>_ZQ�����=��50���,-+��h���+<���Z�wM�v3F?}����#�UU����!Y�U��������<����R��>��:�W����`���F�����D��WSQ������O���|��������:}����'*��Ty<��6���6�a{�����Jn��+���
��#�����G�;H����l'�{I�z���X��,����kv�9t���A��N�uuu�o�>p���������7�������|������y{�Y[�R���P~n.5���.\))+��������k�������������?y��W�����^��4z�K{�����Y�v�o�����O_�q��
B����,��W���_�6�W�*�������
MuuwG���:7Y+����C�o��|���*C��O�5"_�������ZRV�����Z~;��cC��4��=�!��HE�}��0Er�����=6US]}Tp�?���l�����H�Fw�f���\����3P��5�������~���Nh��:Z�'._'�t�����������t��%B��)��z�!^���,�4�~w`(����0���W�������RHEV��/�}(��l��.��3g������/�2�RC2����e��`O�V�cq����w�/�=]����p0)������-]�f��������t����}������?���jcc����p�H$�D�=�"?����Y��=��q_�w
EQ�������h<���+(�{lO��=�'	!��J�"�(��������+�!��]�b���A:�;�->���Z����x�q	!��N��{����|���?���~�{�x_���t	r����k[������������{.A����Hx\��������������d��� @�~���(j��]�O�+�����;���sx�<i�G�g���9��f�����2J�Oa8K��!5��cf�336��r�x�P(���-Q�O�,~id`4�����3����z �����s�s��8k�L��`���g?�_�����;�����P���?��H�,i��<���t�#m�Y��+�%�����8
?B(6 �}G���~U����L=�r����o��LWG�o`�yS���T!���>L}D���&F&]}���<K_W�#I7�n���R�kB&�����i�#��g�O��H�>��i��4���]���!�Qf<C����_����!����+����i����O�|�g�������,�[�j������������x!�r���y+�������o�t�ra�Mu��]��\������C*%��z�M
�X�h�u������-������9��j�g(�rWX9Z����>�?���c6�6m�6�Ia�n^x����w!}�PT9l2S�l�X�M%C�2CZS)_���dek4)�J����9��u�����q�w��vO����R6��0l~���z~���\����{br�������7��G0���]�U���b�����3cS�`�����g"_���u�����%��l��I���������_~����(-����(�L?�R^Q>y��~�}W��OQI�W?�w��s&�b�,�c�,��G��_$��������:6�Y
�e��(6(6m/�����������3�����z;[;B���6!�b����,�0|��_~�W����U����|��7��}��>�N ���X�y���>��i���h��.�l����LCM]��(-/ss��c�V���J�M���U=�0��!��gh�d�b��
����9���u��gS���$3�y���v^�~q��]��/y3��b�dO���~�����_��ng��<c����Z����]���C�M�w!���6�y9�6��������2r�����e^�
+_K���hR������*G9�%S�l�Xe�!y���T��,�;YY�H$j�-BV����s�WKC��E+:u#�w���������:��������K�ch|/����{�����,����-��#'�O�}UZ"����.�������]�;'&'���e=�2���^�D��������o��N���_:�KBH;�v�vOf�,�c�,��K����3���y�����g)<��������=u5uu5uB�:�������x��=�|�}�����I_Wo�?����� UUT�
�	!�����K����J���b�&vI�0�����FN<|��,QZV���������]��'�2D�!����R3RS3R��:6�n��N�~��y����6�������Z�{	�F
�t� �x�y}�����O$[�N�������0|�����q�Z~��k�=��
�7������55�}�.;/g��F�]F�����R�VTZBBD�8�OfN�������f}�p����l��In�n�-�:'?W�!������?����K�yMy�x����7+n�7�55����n��YT\���������2��|�R�
+GK"K��h��/5�hR��i��Z���K��v�������<c���6����������R��We�����?���G_�t���4�)��l�Xe�!y���T��,+6+��!C��_��kt��?���t�`�����>�{���Zde�����a��:�~!
o��}���6z$EQ�R��'����q0��P$bXh��,6���]6���3J\%B����z!������>r���,�����M�C��7m��~���0!����u�^����5�5�{��d��O�>K�>�!�b�U�k�X�b�WM.��\f^��-��#�b
T]S�(-yQ��W��p��p�h�`6�Ma�YC�0t�U���j���]���2��)-/-)}5�?��<y����o�����]��w���|B�e��Y���:z:�����������Q���,e�QZ���}�����]Z�����\9��Gz�����x`�@��-{����OW��')����r����@�sho���{��ike����"����bjdrx{#4��%���6�P��C'������~����,���+��	��'ynMm��esm,m�Y�_����XZ���!�$�<�����_�������; �!���=�������N�)�N^<������>�U����#��?m���W+���r�$Mb(Qr�Fk4)��C7���F���D.��}���G��M��B���q������436=z:����9ud�5.w9d�i-�-�������`h*�k���echRL�L4�5Vn\�f�w��S_G�_�GV�AV|��p�~���G��,^�yH�!��������hi��t�%����H-m~������g)|!��V������<*l�{��G
�;x���
��w�����������G�?������M��N�c���o�w1|����Y�/���^=�>�{7y���B�(6(6o]QIEQ��M��<���<\<L�>����	�]����wo7�.>�e��:��Ui���!�V�><x�������������s=&z��?�<�9���,������Y�B����H�k��kjjkd)��E/�,�����+C;�������4i��OB������8��}G��:�����f��;2rbD�Az��+���d��S�M>N<�������5�GO#����P�����3�H��A�����nk���������"����TUQ]��'z8P�!����+��D�c�)��M�-x=���b��y<e^#�p�X��C��Bt�t��������|�R�
+GK�$�%Gj(�I��:�����CO\8Y'�#����'FA�uM�����-�����t��]�3��ek�KZ�`N^�F���%�r��l�M���AF��
��j��ZKS�@�Y�Y	���~�q�6n��������i��B�p����hi��;�j�REd~��Y
�����v��|f��=Nv�ii��[����;y����{we��|vIi\E���TD�=E0�n91�BA�1�B�&,��:[T��A��������8bvy\P��~�<�����Yw���
]�o������H�l�������������������e�QXT������fU���r�e�!l@v*��L��w���G�
�E�"���L��������
� ��=>u�t�_Z�\g�� �i���O�J;^������BCo]�����%F�>����������R�,������.���������K���������:H����.��;_�vq�
�ot�qm���C�R�o���x�����\}��'����v����w��i������-����[�3L��\[�*���b��wo���!9�u�V%�.Y����s�OW�����5-�e�]�umh�!�N�������71�������*���b���d7`��^�Q�ZC��9e8����y��K�/�������_rs��uar������������/,���_��V�[,��Qf�J��2��o���`f�p8�)EAA��S���EWVjWTW�v�W,������5o�LEYe��Un���#�^CMUE%�S�c/�����=����<��<�\G�I�j�<�k��?-�[B���{�f�Z��o���g�����l�����$W-=�Fa�����,=+�����Zy�g\�WPP���F�K���1q��6P!�5����2��,��%w8���{��x.������+u"e��}�]�l:�����aVv�@ `��T���n�
8L$��&���/��z�t^�B(U��9�O_]TTd;����k����G��:H����[/���z��X�	Q�$7��F<C{g{�#'���z�
531SS�1q����nZ��H��i�P���H�=�T�
EQRR*��B��6��( 8p��@���Kh\�XYI����O_<��l������	
\n]��R���II/!�QY����L"#JD�h
������9x���[9��K��'"{X�0_�$�L�7���h���:��B���g+�/�:���HD��d|�%�
��M�
�J�������b��)E�����U���O5����=��2x���TRQQ(�y2��i���������#����4�(�I������R�����)K.�'QE"�H$�U p��"�,���)s��=:Z:�sY4oy.��X,f>{���a��?��/������vl�AS]�tRE�K��a����p8��g�����y����1��4�W����nbl��e��}y��Q��D��>���B>���v��a��V-�(��]���s����1��)��,C�4mM�G>$X�x���z�W�����We+sK����W�d��t����[w`��}tw�����4�~-z��9�o��G
�1�u�VM4aW�g/�-[�����i��9!$7?�bfb���c����2�j��P�A���'����y��������K�7T�:&/����|���w����O�
����)s������V%�r������|�Es-M���mN^��cT�����%d1*�
��x/J�(��nJa=!��MY0���WW���m���_(������������x����b����e2�����
�y)�A�*YL���{�J��8��)]Y�]	P�U�g�r8�}���y�������7�����K��>���y2*%-���#Q�b<x?���8������1���
������_��B�P$��g����G��#{�$&E�o^�����G&����O����U����������d����R�Vl
JMO������{��J}-�������O�hb^�e+�_r�U�m�B��|M9����~A�?{|�����'vh�����Q�	JR	-�����H���wO}�#!��4�z�E��\�}=5#-����#{�
���Kz�h�����s.���*�/u����gO�(+;��+y0��Mm-�cg����FL�h�r����|eW�zu�}H�8�w��G��2�>}�����I�����=N�?{��@ x�����cWm]�d���W��c�����()���,fQzCy�
����t�J��Ib������%�`V�t�d��_���BL����z�S������;��|Nr���B~!�_������YD� ��,F%�+�LRZjF���$�?����b�r�Rd�������-Y���q���KJ�����U"!$9-9d�v
5u������	!�@,����:�>���b�6����}����\�3�,SJ�B]��]	P�U�>O=E[K{��Yz�=�m&��X���;w��1�������5�����[��R_��K�I��~��������D����[����fjjhN��N�L���o�����c��"������GI��5n��A���
Y�% x���}�Z���<<yJ}-�w��(���S���Z���{�]�0� l6?��m�l\�a�d/-���l����f�,����x<���
\���:�'�hin�Z���n�3����|�����z��WBz�)��:}UH����]z�(�H�(((��ns��Hw/c^��]� ����W/����E1TUTw�	��#x�����,-
�.~[���<��O��_��m��@W[��������^�������L�p�`g�Nm:|�����\��s���oLNM��8��s���q�w�v������D����}��obv��Y�v�	!��v\�)�������F�Y�~{��������C�Y������JvV�����3��H���K��:��bG�S
}80#���ys�V�q3���z�L����w���^���k���l%q���������/]��rx���
�����:�>�Q�bQb������L�������d�R���]	������ZZ��������n��L? �����]&��Ge;w~��i�/���
�\�!D�<]���;���q������B�b1s��CAA�"�����sg��Wb������"���Y����8l������^<�����/�/sK45�w'��-�
,��\.�����<8wLbb IDAT�j^�*o�\n� ;.��g��_���~~z����)V�������D�3�b)���9�72�cJ��p��`w���\��K�I�Ug�����+��3m��QCF0?��>���Q����������\�k�/X9}�w7��g�q�����+�����7/��iV�����o�/����6��7p�����B����j[��<�k����z������o?�3�����g��x%E%BH����!��_�����������{�M
M&����v��1%�v��nq��u������vE�����-&���q��*S��~Wo]�%����:�����n\�������A�C��5ut{w��_��K�I��|��b��eh�V�e%e���S1�+��Z�������YK��*���c$��(�����;��l[���n�3,�D)���"�Pr����-����Zr�ev��
��{�g�0m��"����W��&���78�r�|���������MJM�]�('7g��y������=
����������,�7/�w��U�����E��L9�k�.Wo]�_���8�����d2RW�����.9��WUQET����{�����/
wv���i�����j��7!���w������R�i(���D���/��t��j��.���oG������3��v�u��uf��Z������:Z:����y������WUQ������iY����{/�� l6�K
�<!���1!d����)I������tn�i��i���
��D�SO�D".�{������jaEic�z��_�������8������\����2������%u����**(�^8�y��]?w�A��� *U�}�/��?/��w�|55�.�X,~�<�U�V�#V-�=},����{�:�s>��=a������~$B�o�)�&M�� 9�C���R�L�������p��.���+|qy�y+7��1fj�-���e���
���X�8\B�������Ub��]$I���\1?���B�6��x����LR�V���!��gf�������$�o��S�R>�$U��*s����jijm[�y��0�H8h����c�_� ��5���o��Mm,Zwn��~Z��|~!_GK���Xs���h-������\v���D�N�@����K�m��\�����:4bG�o}��0�s��*�xA�`��)U��v(\���m�^R� �^�K�a���.^�}�6d����a-���,B����%��X����8����i����w��i���~-�����������22���:��L�9�L��l�l[�{�}Zul(�A�����g�O]:}8���Yt���7������o�I�%�$Ju�,��\�^��������nb�xm����}��-q��2_K�����E�;��=iV�n�*�^���t�D��3��Ba����sw�i�*����������[��|9�����qm���C�R�o���x�����\}���r!�b������[��9dX����jV�-�<{}�(�_����p�W 07��PS�z�z
5U��O����/0;{\EYE��������7�B�ih�0+;K (**B�E>K�<~��g��Ff
YT���R����r3�������,��{/c��a!l�<v�{O���ru�u%��~M=%E���TBH�}�Z������U����g4���;��;95?���m�~M}B��aFV�'B�~����J�
-I�,�/�	�����k{������E@T�jx���O%�N�^�c��������
)i��N��>�W�d���i����%Gn��min�<��������ENn�,���D������]��JH|j�3d�i���?�|��iO�=0z�)�J�^�����t�T�(�T���{.��6@���nblR��N�=K�c����+�	!��������g/�yN��)Srr�&�	!��yz�5y��n��#I��0^�����y���C�B+l��a�����vssp9�������<�p��ES�s�O	9����i�op$��y��e��9�#*6:�dTJZ���G�N�x�(~���q�#���;/c.�&I����K������s�X���^��3����<x�`���BNnNNnN��O�W���]���5?�k�P(�D��,�����A�&�e^���%�\�z�a��=d��g�}�=~��Ib���!\.�^�z�?N��q�������O�416i��1��u�w>~�DB�SB��w��cc�]�m��f�}LI����]Yz�Th����JJM��t���{]��UUTo����YIQ�g���������?RB����f��u0l���u��.�������W���o�:�'�hin�Z���n�3���=���I�����sv�?%yk���}��������_�y)��]�N������u�J)<#5=��~-�_K����^�q6����U���u��������������5�	!;��n�<�offv���f��-��SVR&���cT���6�7������3�cl�����g�_��.�j��t�Sz�Th�5���,F�TNll���]y��m�6�P����5�Y��A�+����w��m�Xy������3swI�X\��XLa~�r�qqq\�u�x�
;s�\��c_r���h���������p�\�yUW�7��@��w��&`���L5y���h
����_���~~z����a��ge�����%��_���������/�����6������A�cnc�w;����_����1� �P����}0,`���G���2j�����spod���$#�����NeyJ.�'1
�N^�E"��}���P�i���g�gde������~��?��������[�0^��Z�.=gM�KUE�"�C#v>������������U�U�\���GF$�|����h�0~�8n%��@��>n��p��
���W'/�o^?���������_r�E� l6Ulod��u�K�9v�����������	y�����n�n�SFNTUQ%�3���'�.�v����:O�������[y+=��r\zi���2KHiJ���b=oB�-����������?�' x��?FN5��qJ��7i�;}>s����o���)��_���l<!����SL��`�_����?�BZf���Fk�q3454�~u�������/�P�c����QI��F<�q��8�r�g��b3|��Cf��=wm�q4Nrd�L_�f-*u�S�u��>�H��w�s��c���L��`����/7r*�*���&�����k�H����!�-}	�4/��-����++c�CW��������/�Z�m�QVR.�����n\�������A�C��5ut{w���%���$Vo]�%���������["c�7-[�����W�c}�s�����Brrs������{�����2����
��5��5<d����>��o����YKgs��c�B�O��X:kQ��Of-����:�����My�yw����>-#��gX<�R���#D"���	s'[4k)��������M�s���[�n���o_����t]���!�Z���sq�g,��:d������K�-OIKY���g��l���R$�z�*pc��yS���
�FYA(I����K��HQJHiJ�)�b7oHp8������+s��g�O1���9=�%B�n?{�\����/�w_�����t~A�={8o�i]�Wo_�Y>o^���E��xu�������a[�F�����i���n\���@��C���,�M�>$,q0)%���i�#n��=�)��.�+4�|,F%}���� �G��H��SJ����X���4/��%��r��te	"�H�\����+~M�t���g=]=�j��������v�{Bl
x���	;.��VJ.�'1�=�w����n��8"9� �a�6�oLi��I�6�$<d�?RSGo��Y���Fu���|XKCs��]:�F����Y?}��Iz������}{�3I�]{]�s��>Uw��]GKg���;��?*�D)F��X���NJI��,��{�]�a���z�j��j�����z����$�[����l�L��5�������~�%��5x<BHCcB�8��S��g��(+%��r\����,R�R�ReJ���t�_=��y��#����[��H�y��+�7�����I�&�R���o�7�U+BH�����_�(>[<v8)�����P�!^��3���B�_��3�����8�q����ZP�v����F������#��1a=���P��}���|����n�~��{�-���%���I�-�[,�����D)��s'���w���2��[����#��_3��)%�~?�$1�y}��Y�_����f,��������z�ZfVf����ZXC��
�6����L"��"�]k�������p���q>o;���?����W�{�l\��[�.�E������������x���>�{����H��)e����X���4/��%��r#����!eJIIK�9������c=��)��k�5��������������>gO�;���gz:5��~�<��?���O�'��:Dr��������"���\\G�IL1���y�~�����
W��!�?��~�o^^�sc���L��;7�Y�Y�e���s��}{��p�4�`gw��Px�������5~sd��I���(���Q���URT"��y_"I�b������j��yRo(�^��\�R/@� l~(_�>IL�9��w��2��2�*�d.�K�r9�B�PVJRE�����d������*�W/��]����m�����c��SG�-FYb�Ib����yS�X����
���A��Sv�m�����Kv:�E��m����ZX%�x:a�$EEE�����C�q_[��~=�����>��SWB�a���C�x�)��d��h�
E9t���mk�g/ic�����SL/*L�sJ��|��9��L����p8�
��K��(�T�lf&�7��My����9��-������${;BH�������P"�����s'�N��6zJ��l'���G"F
A/!��f��IdA�(�Q�����[;k�Z�Q��>�b�<|����K{kB����3���Z���gy2z����Z����z��;���������7/��%��rS�3eJ�]������
�\.w���z:5�
��U����Z��H ���������M�}]Vm]�����~���u��O��5��������.,*����\��K�I�+�����u���J������w��c[���?z`���\��������g�kjhn^<j�����[�������������Lu���t5���>IL��9������B���v(\���m�^R� �^�K�a����2?e��b=}�g�z�jm�z+s+�Z����)+���E��������+7����)������R�����y�B(;{������������Yb>-3}���*�*�e�'����@G[�q������5���J��FF�;lc�����7n��6l�&�����l�5j���2�u���v


���J�c�F�n-��!��9�q�am���m��"�wE���#����g�P��rA�H/a��]��|��
!�Ib��s'�?g��gfb���,������Wn�B�R���/��QQVQQV)�.�NCc�>N�-�6����]`�����$��D����QY�p�r��}���=^$("���g��9�g��F�{	9`Z�����k_g�@����H�w
}�.�-Vy�Ao^c���-�M��l�)E����w��/YcV�TKSK��>��
�������]�0H�s�z�������o���m������X,&2?���K�I��=>u�t���w���{���5j��*q�b%%�I^!b��e�L�M�M^�}�?��X�?%yG
�����{��W�����$Ib�xm����}���W��{�#��E�;��=iV�:(�^��d�����Ly'>y��q�PX$(���}��y��8*o�����S�6�.R�B7��/q0���g/����d����R��P�%�y���+y�y�8�t������|�������x������k�����GW�6��:4)5�f����/{N����w�|B��o7���y��������<�29-���:��������J,�~�f���3[�lU"��5��n\>������SGO�����������Ac���=D�bV��HP���v�������^8������S��T�������rlv��.�+4���Q�ZC���2��u��{����{u�y���/�9���0����mY���[����Pa���Rg�2�bI��2�Wjl��|�rS3���O)


6���~�-��R���R�*���aVv�@ (�1�j�**��2{90>|��qe��N����p8�M�]�4�*��������!�������k��e�~k�0w���&f�Y�����b=]==]�����(���������t�HP��t���	{6�nd��{��Wn������Rr�eL?,�
T�~M=%E������|������5����h�j�J�HYA�����1!����w�>p�H����/9_���(�|i��W�F(���zU��]TTd;����k����G��:H����[/���z��X�	Q�$7��F<C{g{�#'���z�
531SS�1q����nZ��I��i�P���H�=�T�
EQRR*��B��6��( 8p��@���j�A�+�E|�����-k�>8��������7/��x�i��m��	�����u�V��MJz	YvEc��m3I�(��5L��V�j^�p��~ko}�dT�.=���am�|��2i������kB�E"�������R^��E"��-�[,Il�'=zl�����T��&��N)����������~�������6��w����5!������S����i����{90Gn��minQ�ac9y�)i)�
x�����o��'_'z/�a���|>������k[���D"�HRT�@�����[��� ��O�^�cX���+�{�=�e��!#�	EE�P(����3?e���GGK�{���b��t�c�����*�_����*���X6�8{��wO�����q����A���
5uc�_�e�[A�I�/�w�+oi�2���"���]����k�������G�����U����2oTH���K����\��u���Gwg��K�H�������������z����Z�l��Av�x������A��z�z��M�Br��!f&f��=����)S��Z
����}�{��P$z��������K�7T�:&/����|���w����O�
����)s������&��Wo]����������X,>}����qR.�[�����g[41���jk�����:FuX?#�^B��.�+:���Q,ZC���>!��MY0���WW���m���_(������������x����b��K���-%l(�K�
��b�i��[�W����p�N)��*�J�j�[]+�h��k���f���>~d�������9�#*6:�dTJZ���G�N�x�(~���q�#���;/c.�&�hj~n��#!�#�=
�
��0o����G��#{�$&E�o^�����G������R�Vl
JMO������{��B8�}�>���<{�\JZ����[�l���'�_���!�<X9?@����������
O5�+�������
�B����E"Y�q���M���lE�K��(��
��G3i��G��-~����[������U{f�WV�
BO�}9�,mU�H�l�B�3���/���2�^R��
9~�����c���C������:v���1?i�D�f-g,��k�Wv��W�����S}g�}t/-3���G�+M�M�4hL>����S�O���^��9H!y= IDATv���L��]{��>� �an~^�����{�b�7�������K7���$&�
X���Z�
fuM�L�Y�!0���%LIO�[�����Y~A�>egy*�M����z�S������;��|Nr���B~!�_������YD�2��,�u�Wt&).5#�cJ��SeJD�k
��J�p �t��U_Wo��eu��th�^RB�~���	!�i�!{�k��K�V��8��O�b���H)�S��!�-VyaCi^Jl����.7U<��2�T(D������X�����I>Z����e��}&z3��(S����f��u0l���u��.�����������o�����'y]
u
.Wr�{�LM
��~�����5����?|�����rK@��������=yx�d��������|����=��6�FNd�"������G�+�P�q������Ac�x�R�k���!��w��Uj1R�SkU�[>r�ev�����A��h�Z���q]��a��t��{w��:z2����P�d_��K���H�KHi�;J�����>��KOe��>��bN��%c�+p�+�5`�������(�����5�vO�����������o�����Y������������j�����4b<���}xrZ�8������M�o\�5���sn~��
����<���]�"����n]������UUJ	��uINM������\�&����&�O!�M��\>k��3!�s��+6�71c����H6K�o^�=��suh;��r���f��IJ9�?�������C�(v�;*���q.�����z��Q���g�.]��[������N�_�Vmf+q����v�%�-]�|�������4m���H)�S��!�-%6(�K�
��n����M�)E.o����8Nll���]y��m�6�����j��e�{~T�sq�'������*���EB$������s�m������n����������?��bB��������?~v�h�����Y��gB�O�]����enc����a��g�����y����sq��3��p�g���~~z����a����X]+����u�����P�����	����w����{##>�$���vw(�KPr�H�}0,`�����6z��!#!N^�/��,�dV����h��|���!F<���6�(dyI���GF$�|����h�0~�8.�"
w��y2*)5��g8n��^��S-�����^fW/@� l���1�?BQ�rk��m���D��z�zL��3�C#�����@O��G��^���!�_����?�BZf���Fk�q3454�\��u`������j{��1�e0s������F�efbV-�Q��	*[��������6��[���f~����7t���2O>}�L���>K��-$<���[SG�w7[�KPr�K�����Y������PVRf~��9B$J�O�;��YK�����;��l[���n�3,^BJR�����Kg-jc��Ib���sT�UG�1��9l���}���4m����+s�th��2��]	)�������R/@� l��~��FO)^�{�1h���{��!'��������|N���$���rrsL�G�Z�����y�M���z�j��y�}�.ZE���^�z����v�r��U��

��~�+,��F���?q�d�����GL�]���=t�HM���fB����~XQ����>�����]���[���w��K����b����g=]=�j�_�����b��R�7-f~u��]GKg���;��?*��������=���'��x�]{]�s��!;~���n�B�s�~$���~������K��(��
��{��Z�g�#��14&�����1%��������m;�3���a�Ht��i�H��ro��=���V-�!m,Z/��K|�������^}��c�����[�C�s�p8�~e7��}|�/�@TlL���zXwG@T����{���v�mVmYs6����f�}<��r�e<�X,?y������V��D
�r�L���pX$)p��s>��=a���/����t��}���e���[�y��i�$�������������$M5����>0;"��@P�c}#�Q���J�&v%����b���\�z�a�#�r��.���5���D��.��^]z�������i�������2�n���=�w0��o?��1V������'�>�$1����2*�h�D�n\������G[��
�~��`�����V�i)G������<x�`��a[��	)������B��������6�Q��:Z��E�yy���Sr�K"�	��?��q���������C#v�x��C��5�m���a���DI*n[x����	���_�v��?����o!O��;��9����]	)������K�z�a�]�z�jm�z+s+�Z�Y�Y��CK���@G[�q������5���2��2R	!<�Z�3���3��eT�h���i7�o��5�;u=w���)���P����~�bq�f-�����|�j���R��hi�7n���+�	!2������IkIN�_�~zVz��p�
�����w��4K^7��kC�GD���tm�:&����'��r�rURTj���m�^����X����r�e�1�?�J;|"����B��HPd��;�4_eUB��������-gN�yAGK�������I��7�o^�~�s�������s�
�a��������2*�h����:�:Z�[���#*[�����gP��X�M���2����z
5U��O���:��D9v����
�CjJ.��"��K44m���%$�W�r3�������\��HP��t���	{6�nd�Pr\CM=�wE������T�X�e�qm�������Ek���1q��6P���'y��r�:���v�~M=%E���TBH�}�Z������U����g4���;��;95?���m�~M}B��aFV�'B�~M�_g�e4�O�9�/�������1��n��7��Py��
�-���������_�cX�v���p8-���}����u����E�'���&�~���#K.vI���/^�$IJH|j�3���BN_:��MMu�oo�@0e��)�m�Sb{���k�O+)*i�k������vl���{�RB��Q���{.��6@���nblR���F�mx�c����+�	!��������g/�yN��)Srr�&�	!��yz�5y��n��#I��0^�����y�eT�%��a��c��?xz�����h�~Gp�	P����`g������R�S������w�����9�#*6:�dTJZ���G�N�x�(~���q�#���;/c.vI��E��\�}=5#-����#{�
/����<h���D�������_��B�P$��g����5<���+�(prrsrrsr�}iJz�����������Z�);�c��J�&v%�����{.J�a�����2�����A�?�$1����.�S�N��������^Zf�����W��4i������;?w"!�)!������1n���wFS3�>�$I��~�����RFSi*�*Nv����w���)i�Y<��P=o�����rK@��������=yx�,����������`��u���]����SW�/A��.i�$-��~A��2�
y�>�8�����Z�n��4v��7/%�������}�N])IQ�?}�����$��j��'oB��uINM������\�&����fX���������`�_r�E�f�
��Gcen��~]�����k����l�g�.mMmB��5�vO�����������o������	!�������1�
�
2�2�m���[��#g����qQ�t�u��2Z���0ob�t�"D@%����ZZ��������n��L%|H���_��5l����k�}�X�6��wq]]���;3w����`�����r��������pv��������|)���&�a���A����r����=n�����HM����:j��Q�����[�?=l���������/����g+sK���_y/$ys�_��o��mnc���������2d�v��_
��/��c�A��X-k��������G����]���c������d�3�2��i�,�B��"i�����+�_��)����WGjM�|�qC�x��m����;��LNM���v��w���*�*��JRf		!���GF$�|����h�0~�8.�E�V^/�� l6Ulod��u�K�9v�����������	y�����n�n�SFNTUQ%�3���'�.�v����:O����������o��-��VC�W���&����"5���.p��/	��������H�zE)!�����7}�"p��>��!


�)��=�+F�1r�����Sb��Is������,�����M�<�Z��g�	!�]��`������6d����2�5�5zX�x�����Y�"
w��y2*)5��g8n��^�vh9
%w��..6�+Z����k���q�#g�Z4kQ����,��������F�efbV�`E��r#������oB������GI���3�-}��4/��
"v�������]	�+��[��7G�DB���N�h����O_:�q���+s�	}�����������\��>�|�h�24h��%����V�����w�a��m��2��x���5<d����>��o����YKgs��c�����R��S1�+��Z�������YK��*���cdE;��z�]�a���zNv�w�������q>���kT�!D[K�r.�����C]�,��095y���)i)�����������S��W�^n�4o����_��rrs������{�����2����
��5��D�z��/�_4��ey	��HY��
=v(%�T�Rx��%�[b��ceny����#&r��b���D���g��[����%��+s8��/h��g��-7�k����9����]�hUet����{#���^��A�K7��
X`�g��U{Y���Pr������}H�X�`RJ�Y]��G�\+{�Sb���e7�T%S
}�������3)�7d���3�-%6(�K)e�����,A$I�����t%���zn����������M���;y��=�\���&���n��v8\����\��>�����VC�B��$9x����3�m�����?��������t���3�Y��~����$�������#{��i���k�kw�3T�s+����6��������Na��j��o\�H��%l[;���'�"�4o�LO����{
��� k����!u�	!�|&~LI*���:t�HM=�Y����~XQQQjRy�=�w����n��8"�KPD�"�n��C)!����S�/���z�����G�q�u��d����W�o��1�]�M������o@�V��6�y�%�~Q|�<x�pR�Gm-���C�
g����|��Eg.�SUQq��������`��U�?��.�k��7G��cr���!.��:v!����0>�H�,[��5!d_���1�}|��[�_o��cKo���2t�p�����>�$QJ����L�>��������|����g��}������[SCK[K���k�Y�m,Z/�ka
��6d7*�
X3�,�v���5XL)�����p���q>o;���?����W�{�l\��[�.�E������������x���>�{���^/��
��R��Fy�E�
J�R�AD��e�v%=)SJJZJ�AvGwF�8{����O��]{�Y������5)V��������j��y��AN,?y������V��D�[�Qrq9I
\�����>gO�;���gz:5��~�<���{S�S:���X%E%B���%N���� 
�����������$w�N5�?�a%}`v_*�������3R����k��'�	3�����V�VV�Ve���p	!\.�Wh��wn��l�j���q�4�5�����6�Y�(Ie�����N�_�6���P��)�C���]^�)+�,�WEE��nk��~��];v�>u����%��$&����7u�u���X��
b/���n�lUvh�^���/����m��n��*���	s&)**z�
%��
���Z������<vx_�����Bk�>R�M��%����(�T��"�:~d��������h������	��9�x�>��If&fg�r8z	Y�������B>}4a���y�:���1%������Ac�~��#�����r�����}�����%������QCF�K�nT��,f�(��5*cJ)o8������aTl�����3?b�3�����-<������n7�F����t�E������{��~��)��l��X���y)��"v�rU�l�)�v�����6r��]k����,,*DWVAWTW��]��C��5�m��*������B��������6�Q��:Z��E�yy���Rr�K"�	��?��q���������C#v�^��I�g�+�l�N_�=}�T'��eL�#�%dl}��0�s\E;��zY.��
������)�
���������!����k���NNK9z���������{m��uO���2�����u���d	�b�E���+������vJ=�^���,�
[��U!B����������g.�-���i���LVQV�-�<)���:�:���=�����xW��72��y`��\.��qs�a��0I�.��g���QSe���C$_�SPP0�mT��5�����9���B������Nd�f�P������8�������ln���u����EB���j��X� �����E/���&�&�D�+7��b���q��������pT�UT�U��7�S�����!DGK��E����%d7*YX3�T��b���Rd\.��������E��@p��17���^�w�#!L��SVRv��,��S�{��~��)�����X���ye����[������S�~M�W�^�_��������~M}tet%@uU���[XT������f��\:-m���u�u�b1��Qv�\MZ�0HrB�z�������
 �:����!#��v�u�v@�
�P8���,IUI,�
]�o������|{�����B���T�U��)���'"��9.
�E6��/�6��G-��`� c��o_��90��OzRi��=>u�t�_Z�\G��HY�*4���n���_�`����^>��O�H)!��RO��*���+y�y�8�t������|�������x������k�����GW�6��:4)5�f����/{N����w�|B��o7���y��������<�29-���:�����-����?eg9y�*)*5i���k�G�n(�X����anH�l��U��K��_�q�������
��Y]�"AQZF��[���{�tn~^NnN}S��m���*����Jv�B3�,(��5���S�C��.�wo�t�r�.=/\��%7�_&W����-k��{�_X��p	!��������le���e6���(s}��\3���O)


6���~�-��R�����[�Wn�������^CMUE%�S�c/�����=����<��E.��"��K44m���%$��TG����D@OWOOW�y�f*�*+7�rspe>�$U�"A���9��'�����Y�o���{�a�_SOIQ)-#���>_r�C���'y��r�:���R'��3`�'��ob��������T�"���+c�M'�uJ,�e.R������D��l�~�����G�O�e!)!!�Re�
K��***6����v������1����u�����^��w����>"��In��-�x�.��.��GNF���pjfb��Zc��	C��&�X+��{�B�PIQ�H{����z���"A,N�W IDATQ�����j�Y�]^CB�����"'�������7
\� �9B/!�^6�m��������/�mZ�a[xh��.��q]�y������������Vd&�%�X���q�JZ��<�o�����{����|��H=��k��{����#R;��bG�ED�* Ml�	�D%6D�D� 
"���b�����w�������;;��vg?��l���c�\�75M��IY�aYMM�q�ct�sy<�1}D�Bd?��3?���?��y�%��gl��!�A$�i�i�l|>_�)E�����M���Ok~�{�����))(�l�`0z�}�j;��Zs��]#}C�'�e��
��5�J�J���,h��-���5t$������#|C��8��T���{s�I������	���\.}P�p8K7�(~Wu0\EY����������B��/�`0����q����Z������"�w����(���^�;�C=��)���|�y��^�hiS�ji�jy��8�E��O^�bUuUUU��������mL�N}"� %���9����7�o~�#rZ��r������X��f�y��"�J����{���}Qk�~�����
�Z�}�[��`h2��}���}zt�!Y62�g���o�.uUujM�z�������.�]�2��+����k+Gak�����Z���2�e�Q���q�f��r�Q��R�|>���WF��@�UT���^>l|����o��6�����s���w�S�:�^8=�����5�/me&������������������/�&uh�A�o��P��R����$b��Q��d��B��qvK7��~�}���Gw�_��r<y�y�F�=��g_�����~���dNs�E�mh���o�D����w
�F=�1��4e7%@+���������=�En�j���t6��������q	��\��o���)��\M�&f,��![�n�y�V~Q��K"�N�M�)fq������TQ�����ry��������eH��+\�+��������c��a2�4A��@���r$"�����7�l�lSV^VV^V���������W���)����47�g-|�����SZf��;7=�.�g ��{�8�!'/���_~a~�_7N��;L�TK���W����c�?wWM
����z"��s@��Dz�0��[��9��C�"�g^����$^�EVF�v�����V�-Y���$����/2�e�r��O��$�F�����]�y����>�x�3h�n{��~$���w��Zr�o�8���/f����H�|��s�.<LT^�1(�� ����+��������E%E�~[����L��5�b7�l��������|���~�����Z)t��y�/����^<�C�Z�s���n�6��`B����.�~U0vuMuUuUUu���p9�o��A�C	�J�X��$��/*x��#���L��$��9��y8BF����-����;�3P�C�����,BHnAn��Ey�l�T�WUWB��v>�O_.������dN�U_���^��A}�i���l��R����)��)Z����o~a�����&��:�{���a���tl�q����!#�&�dAk{)+���ZPT����Z�I�A"Nq��6��������C!v�2b���,e��~���"Mu
S���g-�6�	jp49�L�~�����V*�V��/���q��%+�8�6�6�M�~�w�wl��%n*�,��f��,A��5��w��{*>���r���:�UdP-�R�lM�`QQA�
�)XC���H3HIv�I����&�"O__�l��1�����
��i��r����������m�L�M;�������;�n� me��?�?4h�����eE����m���>fi9�������#vnRe�X�Z-v��"�9��-�]���� ���$���XO���]�uu��r�z!{��|�O_Q��&�W|���+7?����j;e��9�'2�j��;7Wl��=�V�-��$k���].�~�d�PB����}�u��B��4Z��$0$(0$�r|�������9*%;`�?�|n���|8%��*K��GI�aO)��u�M��x(�������SoY�y�>��S�t���y�Fm-m�Q��$���m���}�!������{Hv�Q_����h.�h�M���
��H��rS���<�4�%1��;�HJJ22��/����3����!�1}|��lmj�4���rm���i�>h�����"��.@ss�����l/qUU��C�R����|�|>�B�`2�)))L�@K'�*h�<Z���BHj�_��7,��F�j�f�~Z��W�n�5�$��0�3��0�3@��G�-��xx���I��Rq���'���XTR���e3�j���RR�6�\�����y9��:N�&����X������}>��R�����f������A'c�v�-���s��vr�8|>�Xd�������,e��X�%����6^cUVU9,pj��9��~��Qgc"�#s��j�i���x�\���$�Lp����s�-�t�|����i��ES.@�A�iz����>!��a2���M]�����N����x,2������V����<���	!�*�������PIA��d����J�JT����O��|������b��4��Z��]���.�H����EM��VNp���q/G%I6��zem�KNZ����'�'�=�s`����fg���`2�3�������w�����X��a�#���TT�G����&�dAe�en�gY�4��|}���
;7�:����>�}��^���2&#-�59<~����~�������l_�d2W�[�������G��G�
��u���&����}��~�}�d������L�9��!��"y<�`�����2��oe���6�6��83��s��p8���w�[�a���0B��kI^;��;���������{kYy���!�~?=�|�k�O����_e����a����{!	Ig����d������scG��6m�����j�&��G>�P�W�`�-)-���u�1Vme��4�����a�������GBH��=���0�Q}�<>�f��(sB��&���{ag"D>���%Y���85����B:���rF�M����U����*���J[Vm>h!���6`��,�����e����������'���W����c��l��{��-���MhBHB������?��W���d�tt�oE��[�M������k��������F�?�����{�x���.�x<&�y��]�qS��6&��3����;��3*�#��c��:��b������`�qv�uM��i��u}���3�$a��H%���wh��q�u���~��Z��~-G[��gJ��''�Y��{�-�y�����<M����`�qo��i\�&}�X�Bn��=�������3���w
�;��74A����{_�p��1�,��9F�ERAg�q����hB��M��F�_7l�Z7R3]�JJ93�z��AKg/^��P��&����f�m>V|�}h��y����5M+K\.@�A�i#�Ia2�*�fg�<\4v���;�L&!�[�nI�%�x��
�g���N�����Wo^�0�6�$� �m^N�9�!U�Uvyo^�Q^^^�VYI����S��<����s<�_����c$�:�N��8�|��<G#�~S���37�jb�[~�TQU]����x-I�QB�YeVuM����4����,��[�w�r����!����n���	�vGT����7�r��]'[O�sd���P�s((�� _c��+�x���l���Fj&�k�"�8�$+}���Z��NGh�i��+2�o�)����|�������u�tJJK!������r�
K�v�D[�	��w�>��Z_P�Oakh	��~���)(�`?��C��x��t��i�b�k��M���?�������U@K�:'|>���I���z��3+;k�O^��R��~NE�����*K���B����&����|�^���iu��>�}.f��\� �-~�d�v�ZXRv&���kr8��}�����ww�r�\�qv���|~��������j�Z]S}<����k�hz�oeh��m�Ng.������rk85������++��B}:�l���|����UEY��v��{��������o�~w]�>�f����6������m��S]i�i��/�	�m��"/'o�g�$���K���*�!C�|�{<���s��	!�=�]�N�?P��i9�������m��P�6]��(+#� �@�>M,�!A!DK]S����]
K��<^&�&�V��w�^RZ��p$�!���WWUWWU��������C{�l&3�M��x
��k�������O���{��?�L��������['�f��c1�|����Ta�*��j��R�����kS�����~|����������3�j�$��q�$l����n���!�zc�RR������-����zV����?�
R�t�rD|���/&YMH<y�c������uN����xB_�p8�z�\2��w�
����H�P�Ig����������%Y!�P�����8������vQA�wo���uA��Y��:RRR�e����N5����HIIq���oA��t��7yo����r�����))(~�E�����B���
��u;�t~��`0L���5BK��C����:TP��L�e���6��C�R^�Q]U���u��=A��G�j:��m�e<�z��������&F�;�?� �d���������j��1�,�����L�{�s�J<�h<������eD��'Y�\����Q	�����m�j���t6��������q	��\��o���)��\M�&f,��':������/�O�����p�	�"�![�n�y�V~Q��K"�N�M�)q6���eH��+\�+��������c��i�W~+��*>UT|��r�\��y<B�����O����
�MYyYYyY�?������=��L�K���c���m�m��9N�i�i����=}�$+���_	!L&�S�Nor�.�������e<��[��n�n?2��IV�^L�� ��x��l�9;���31�������eS���z����C��;�jjh����������$'?w����C�&YMl++����Z����V))*���,,.�P��e�1s^}�:�{���a���tl�q����!#D��&�dAl
����w����b)�\���Ns�v����o��E�:l�E�Sm��&��,e)�|���i�k���^<kQ#5�����^>,�B9�#p���	����Z:�
B�������`1�0_Kh��i��ES.�q�m�m�c}��������<SQA��Q���,%!������A+�W��(+*
4l�����!d�������9jjh�Y���p�/����s�S���TZ����d�+*(�a2k�a��uvx��>�dIIIF���8}p��T@-or��]��W��~�}Q����.�0�m��3o\PUU:t(5�$���A}.���d2SRR�h	t���b7��W�]����CeU��!<�����S�\w�3b���{�Q!�
I�
$�`0��^�l�T|sx����"������>�9������/@�'�*K��r#"N\HL��������>����f������#sss��u��9N�l/�^hb�����F���UW�����=k��$�������s����s��Y���-���%�DF��N�c�Rw�X���*ww'm����/W}�^�~�����A�����Qc�-[-++�x�Uo���C'����������9:����W���)����4��f&n�s��gR���#������w!�n�O�L��2����������� B��d��:g������0V�2\.7$�pRrbqq���������\))�������p������9�i9V6�f[����gEG%t���v$��7����@�������R���=�i��~����?~��I]]���un|�����M���6JK{�e�zUSS3�]��J�xn�����[���ff�o�^'+�v��,B����1�6m��[��_<��^�d0-ZN���������#�]HK�P?hbUT|�?������Bm1sH���l��Y���kV�/))��m�������h�f��M��D�x\���<�����W���d�tt�f���z��99o��b��s���o���T������������}~��,>�j
N3V�2!!�/$�����k�g��V��`0��g������>|PPP��c�&A������r5TG��|S^o|?Z�[���.N�s01N�4����������FE��?y��9!DK����{�1"��������3\�����������-������Z�MC���
2y��I%���{55u99���E�~�x���4������'fib�;����b�BH��#"��|�Kb4���C�W���������oe���6�6����|���;�4d��U��������bB��kOY�rQnn��N�V?V�2ii��������1p����G�;z��~J�uG����}��L� �X�HZn��������e�T�����f2B��
�JJ�����o�,))66��~�99����.3��� 4$�cG]��D������~���~�%����p�o[kk��x�\�[�����t��N�5�����y<^�9��5�b��s����B���\.�����o�X�d%���������<�dee����X��2e��X�qI������S��a}�!w�����_P��o��***���tp�NS_�&4u(PQ�1p����D���,Y��������j����<6[�V��&��V����Cf�X�C�32�t��c�����=�"��TWWm����fc���
r,�� �X�Zn���bBC����khh��i�*��RRR��Bded��/�Z���`����N�Sbc#]\��l�����'���E��h2������y���dD\|���/	!����.],--�s������*�u�r��!F��gY,VMM����4�����c���g������������\�p���xj%��y����������M��7,<Td,�������X�������DE%��{�fLw
=r�xp#5M6D�!!$**B]Mc���"Sh�V��\�n�n�i��	!�����|�}�P����6��L�-�X��!3�v���8W7�a�����[Y����H���G�8pHC��%H3V���������l���%,�t��)���}�Lf��z3��b� ���]{�	�K���o��tBH�6mtt�}�G��+�#�������~�����_��]���dii���=zl|�i�(����=�X*�|>�Bb��E���:x�L�����u�c���[Y��������r��&�B~��'��s��E������@��krX'>������o��/^f�������&�HS�55���'��X�`|�x
��������T�����/!������D����>j����j����9y2������a=z�|�,k�&/ii��s��������.����d����h�<tx��O�V^�x>+���4�y���~�*$�HH�������L@�mNi���8����Huf�Y]o���������m�555UUU���AA��}�Q^^^VV������f��!����DYy��E�������.##������v����������{��%++�g�����p��K�w�J8�J�����\�MMM-M-�b��]���D�U�h��r�VJeU�����I��Z�!t��������������eN�[B�`�cj>w���TQQUl([ IDATPhm�(r����9y��un�����^���8P��_��}����%��:�e�	~�X��8����)+�������i����EONNn�����Nu����1�����H�e��\o�&"����Bn�y�cy��1����|>���W�Y��CO�����6��{�w
��<�����ii�������p��]QRRr,��R�����O0�j��������������?o1|�����4C[[GJJ�>V}�)W������+���n��~����n��������kW������y���W���o��������#X���kM�gN������K_UU�ZTPP��A��V�����C����x<�����{�x�����Y�6{Q���UUUU�#}wKp,�$h``$�X�����&'�����i����u��%=#M�XRR,''/''G��[��u$��fr�����[l�?�lIM�[\\���������m�����x�������s��.^<���"���[)��S~�~M�X�O?N{��O;�m����������B���ex���~����w�������9Fp�s��m�u�����_.���rv�)2VUU��O�>Up�\.�G��n����&���C^^��@������oDF�O�l�H�D����P����z���L�K���c���m�m�	����v���W/����/N?EZQQ������;�7x����d����ZD��u2ff�1��23�y<^v��3q�ff�u����������1�������f���^��$;�i9��IO����9��r�����l89�\��|��%�����%K�8@���~��H��o��
�����~�����������a�����^������;l�H�5OE����i����u��LF��M�3g�KKK���
6������mB������,���"uu�#F�����f�
��G���������Y��s��_���������\����C!�w3A�C�X��Z����>���rrrqvvm�f��MR

�5�yi�iZY�r���n�n�M���ZQ�1`��w��ijj:;��&4�o�����.��+����ld��;��ce����%������,..TW�055��>����������EEEE&�)X#��L� �X��ZP�/���s�~����2f�ye����#��ob�T������h��D������~0������/��>�hl�7.�0�m��UUU�J��������|B���d�����j-��x"�b�g��o��x"���D4sx�����_�O��������I[G�o�~��q���OG���hk�L�s�<��&�/�X�Xqq11����o��5,-mf�Z�d2	!|>?,,������\ee�����y�ddd	!�Qa����X�������<9;��pP'����g��TVV:O���r���54	B^�~�����A�����Qc�-[-+++�V����Cqj�iZY�r�
�M�kf��:��y&�{:r��=�~���������d*+���L��j�����
"�0�L6[{���s�x�������|���\.7$�pRrbqq���������\))����L� ��L��)���z�c�����N��4v���!����R$ �^����m�+*>���z�P`aQ����p��k�|7m�f��(-�����UTTMM��L��6'V��s{�}7n�jl�733}��:Y��3]fBN���9�i�O���������Z&��h�rB����
��BZZ����q�Wy.��7���>|���� XC�`Yy����LM���^_RR�m���'����A_�M��4u(Nm4A+KV.@�A�i�,��g�^���6..f�<�����NYZ�x,X��p^�����z����a��>Ah:dH���g�vv������+<L��Y�p4�2M�4#l3��T�/B�C�s/��H���k?m<KYe����SK����?~�SsB�������c"�{��E�����f�p57���F���_��{���Jk�6
2�
4��iV&����������?���K0��g��r��	o��������3�����I���8UU���B���qF��M�6e�����h�V��\�n�n����w�������AC��_�/p����\[9--6!�]����+������kM��s���ii��������1p����G
��c�&A���T]�)WTtxB�����,�������L�AY��SIIYYY����%%���}���"''����ef���Dv��K�C�s/��HL�]����������$iiiB�������|~FF��T'�C����<����_������_,�q��7��B���������s���K����|�����t�����LUU5��S�L������������ |/���j�vo�5�_�gNE������O�� ���_UTT2k��0�
��V��	!4uH_�W��j�:���@����j����<6[�V��&�5�����C�kq�P`F�����~���;���[����?��K�f�m&U�L�zt��^CC��O3Vy.���r�6�"+#��|q�J/�K��/ur�����fk����<)MM-1s�����R$��o�Q�V?WQQQ]]�b�._N<�(1�,���������+7� ����22�g����s��#��[��p�2���J����;[�	�����?hoXxh����"��4F�+�������}Rkc����������	�1�-4���������-(N�W_�%�2V��w��6[�RXXPk����k�[�t��� 4r��Dk�q�n����sq���gk;�awA,�� �+�[�R�#�7R�}�r	�=9e���q_&����������*��`�k���f!��R16���4���M�v��	��K�����4�" ��<���2�gO=��*��'���6'��?t(���(�];�
�rqq���MM������r'M�#����O�A��]����#��������<�|���^����^�T��84	��|=����t_�����qw���Z�������_�m�mZ�O����/�B���OJN�p�N����V{nhM��Ah�<r�JR���=z>{��q������9p4����a�s���O�*j��x�|V��-�;����\�^�~r$$���_JI	f&��6��m�V���irX����\��,���������lII�����AC!���22�
�
_����jjj�����H?r�d�������|55u55�=z�����0~��3v������#x���������G��p8�wx�Y�QAAAd�'������%���Kqq��	i�M�Y
�%�2����4CLf�#X�����jS��99o	!����15�;���`����s�mA�hv*�������
	!�z�;:L?p0��G����cYd�b��N�3x<�������0��1����EONNn�����Nu����1�����:��-�R������~������3��~���
��^�]�:o&��qyyy~A��[IQI�Xg��%%�!��Y,A:|>���v�..���}))�K������G
6�Z�|����#<���kW�����(X�������6m��������,,G��
044�IP_�������|���W/�l�fu���:��
���ac�
�M3������#X���kM�gN������K_UU�ZTPP��A�U�	�B����x<����`6�.�;�i400�a������&'�����i����u��%=#M�XRR,''/''G��[���64���)����~����{�.���Qs6���e2��N.k����o0`��?��y��9����#r�[�R6nZ�q�V��w"c?�8����pf�6e�e�a(**233�������z�������GG�#���s�6�.]����Wl��E�g�������kz��?��,X<w>������a����	��9D�D��st����,2*|�tW����-X_�����������C�6 ��S���[���'��g��/�����y���s�~j����C��� 4���2&�T��������/��E��Y6�.�;�i9��IO�w�U]�����@FFV�H=[urt���n����#M���g���}���\�E�����BE���s/��HL���������_7w��������C!�w31l�H�5OE����i����u��LF���m,N�3g�KKK���
6������mB������,���"uu�#F�����f�
��G���������Y��s���WP�������������EEEE&�)XC�����������OG)+���\��]E�jS� M�W�W��j�:����f�p����o��w�455��gN��7����K����z�62��=�	�B�!�.Y����n�gqq�������,�y
����e�iF�fRuM_��������_�����c^Y�)$�����X,33��s<��BD���s/��H�����u�E}����/�7�y�B��K\UUu�������|��>�O�~0����L%
����/@�'��/f{h���/@�'��o\�A�@3��~Z<<�h��Zz�������ut�v�?�������#sss��u��9N�l/��ib�����F���UW�����=k��$�������s����s��Y���-���%�DF��N�c�RwB��
	9���X\\���ean��6WJ�m���K??�R���G��l�jYYYB��������	v��}VxMee���)\.7!>I�������Cq��4�,q�tt�&6����u���Lj1�t��}{���!���1#�	��TVf��>�����Da2�l���AC���PSSo5�B3���V���:�	�e����H�m�RS�z,����S�&�Q�g�����$#�����m�+*>���z�P`aQ�����A��v�r@���M�z��=��u���������/�����q�Vc�������������2�r�Dpt��M���������2�E��B>|���o�?��`��2��������������Y+Vx0�����B���.�ejj�f�����m�6�<yl�BHhH$�����s���A�R8�������i�P�r5A+KV.@�A�i�,��g�^���6..f�<�����NYZ�x,X��p^�����z����a���4��h%Y�|���X�I�&�������j���&$������6���2�a���������������'�15'�hi���/:&B�]l�X��~5c����4z�����E��UTTZ��i��aT��A&O�2�?|x���.''�����
0�{�	!?��c��!ii���sg�TU�W,_Ci��cD��3���&&������$���G�SR�;:�8w>N��&��lA�:Y��ie���6�6����|���;�4d��U��������bB��kOY�rQnn��N��Qp�q�f�����9�Hv,�$H��F�r���p&7�-��bo���4��`B�o�TRRVVV�}�fII��q����������u�Y����!�;��d���&��Q�l.9�T/.�`�}�7>.IZZ���~��A������t��N�5�����y<^�9��5�b��s����B���\.�����o�X�d%���������<�dee����X��2e��1c-
��x�����/��u���y��Xw�����_P��o��***���tp�^+���Y��A��Wuu����^k6f���,��$�-HS���j�V��\�n�n��Y[�_�jq~~��]+�jA��j
K3��7ZI���n$>��K�f�m-�\qq1��Gw��544~�4c��b)))�i�	!�2���W���X����R'�)���..�l�v����������<�MH��r�N�Z���Z***���X,���I��%&�e�X55�?�7H�'�ed�����������G�_��s��e6�������w�6��7i����P*h��Dk�q�n����sq���gk;�
���KJJTTR��'h�t���#����CTT������c�W=z������Cjm,N�M��"���r}�V����
�Ms���&��Z��e��C�����37��S�h%A�o���X�/A�V|�n��Gg��U.a��#�L�76��d2{��sp�C1�v����L ��X*�F}3��B��i�����?�� �e���&$&�*�(+�z��c����|Ba0�2��?t(���(�]��)..�VV���w��}�\���v���w�	6���kQqatt5���!��$
�����gY7yIKK�������^oj�t_�����qw�+H���:2���kByKOO�z�R�o��L�)��a��j�V�f�����O��|	!���'%'r8\�f��Q�=7�����;4���p��F�cYd�4y�����?}�������YYO�x������E���W!!GBB���RRR������)m��g�y���\������G�������%%�6�
!�$''���*�+|M����-�22��9��kw��|>_MM]MM�G�^�����{&����<�]�t�����HII��<��:�woCBH�^����n�jjjj�;�"!�s�.��E�o���RYU9b�h�6g��5�7*(�QF�	6�:��\M�����@���6.G�X]U-%���-sr�B_�cj>w���TQQ������;��V_����	�e�	�9�
89�����k��><x���4g��5�r�����;g���S��u�g��~ut��d���&$&��o���oCa0zz�O������K�k``(|G���<� OK����$N,��v������c�,�� >�?~��T;G��������~��y�����
I-f>�������s�<�'H���0����������������^������v�J���k�<y��y���^�buuUUU���H�]���"l��!M�$n�����
�GUE577G������B��"����^��UU��E�t[k��7���V��p1�	�e�
�hF��(+�j���y����~Z��_.z�:uI�H,���������D���/}m�\����Z�[�UUU��O�����9ee�2�L����qvrY�neo}������������������l�������X�O?N{x,8���MYy!�A������2<���n�^��_��*����#�r�s�6�.]����Wl��E�S���,cbO�����z�|��L\���%dg��/���aFv������]�3���CKk�5�z��MH,�;�\\��0UUUql���i�%q{5x�/�~�n�n�,,l"N8`p�n�32�$'_�=k� ���"'�mnnN\|������?5S��C?Z���b7�4	�a����[�6-�\�22��E���]���f2r�H��o�Y�au�~+Wz��C}�W������	��x�~����������\�8�t!d���a&#h�>Og���^k6�������}�u����L�Xg�D���N��l,''������K��Y����%E��#F��7w���^G���gkQQ!���l������t�*%E�u�=����5LM�g����45����=}:JY�������*����|����&--���,***2�L��	6M��a}����<�u?@�A�i3f�VT|�����w�����3�	���|�b�����
�z�����N��f����&�� IDAT������2}�4ynX-�\��^L�������1����BB�l����R13��;���j���&$�HJJ�:����������q�?�FJ������C��>�������|B���d���`*Q��~Z<�~1�3@���~Z<)B�Re��'N�i��5������/4_�	Q��V5��B���I��TVU9,pj��9��~��Qgc"�#s��j�i���x�\��Y�s��s���#����c�8Mrt�`/��ib��w>�,2������\�2�v���Y�ded�X/�y�c�O��Ty9�����Y�����8����
���6���^�J�y�6���6�����[v������_�^
Kd��
�MS������	!�
����m2`�"7uUu*����c�!����T��2�\��!-%M��Tx�����J
��&�=�TRT�b��?}"����m-m;g�I���;���.�u�DB�.-����a�>���`04�4��]>wY{�v�6��Tq��a�E��4
b!��������Co���T�:������<�����e2ad���Z)_������Z�����=J�?XPT���#t6��� ��k��3��$+}��ume��q�Ug:��_�u�w��6c}�����v�WSQ5iF�w�X4{?~����~�������l_�d2W�[N)+/s[>�b�������o��9���En�	�T���#�?(�+��y�6��>��q[.\������}5x+KV.@�A��V���,������~������
�N�B.^K�������g�uN~���[���6-�@����q��]|:w���*{���
�6l�CIH:�u��%��4���;w�i#5m��Z9_4L��������_F���!h����7;�.Z�4.8��`G9����6��i�K�X��O�\��j���Z�}?�u�h4���~�����W)+)�hi�L�\~�k�2^EYe���{SK��z��+wGW�1V��&�l����n�������)6�-F�B�4�w�;!�9
M,��++*mY�i��aT����g�T����T��.^C���cB�)))�����������>�n���q�d�&���oBB������9H����%+����|+rm���lBH����^������nw4"xh�!�?�������%�v���1����������1!��a����Y/�Q���k=�y6��?��D�������E��a�&��o_G��IH:��c/S�Q�EGKg���s<���{�A�� Vaq�����m����i�B,	b	\�����A;����Cd,m�����AE%E�w`2���Z����QI��{���
Z:{�����7�������'O��';	��6�9����LM�f���\.����)�X���Zs���F�����rUQA�����n:u�&A�J �TUWm���y�F��n�3O�����+>�>�g��
�O����%+����4L��d2*>U<���l=I4v�����P��u���[��(3j��A}�3�RX\����.���
4��%�m^�\��`��/V�x<����G&D?|����6b�	�y�y��II������~�k�k�B,�bQ�d�{�X�a�:�C?����"���s����c~G�T�pj�����@=&�h��'Y�]���SEUu��2���$��F	IgU�Y�5�?���)~�:��3������-�+�-�`1�Z�[�w�r����!����n���	Y�J
=������C�4�u����`��
5
�cE����,Y������~�h�o���SRZB|�����TX*�3'��L���#�������|B���������W�
u��\��b'k�������si����=����=x��~}����-��|n��e�4
b!�d�!����-���5�kh�/!$� o����g-�g��Vh&Z��t|>�?x_��c��t��K���2K�G=U�*��'��9i'M,���vr�>|j����A�1�O�7��{��,����#�fN�}.F��B�2���_�\�R�����f���5��cO�9�~���
����|t���������10�m�����?B��L[B������c�P�J	!����{�rT�,G���"�e�[��Da2�B�B_����[X�H6L��%PU]�_�o���H�H��OS���r��t[U�j��}�����6��5�n}��X�%Y,B�o7�0�� }�?�����p8+��6�3t���,4R��`5���������?�C���m� '�VV��]��X��Weed��I-~,����|uUuuUu�z����>���f2���R�~���n���"���bG���P�q*~V���/�����3�����d�8I5R+C�;��m�g���y0�L�������.-%�_�O�6��o���������������:��&N���K��]�]��k�iB�7�)%��!j�-�B$&hb}�[�b����1s=t���4��b���w�aQ#}�g��{U�W��*�" v{�(g�z6TD��]�)`=��yb�S��"6:��H������}�����<>�l&�L�%�aH�_�w�S~�qu���E�������<�{���sGOK�r�Na�'F.�b��Gg�5^>�o���=#$:T��z����Y9Y����_�@��c���������6�]h�1�D)��1kev��/��}������Lg��O�/�|*�`.I{
�}F��9(Z"++���s��y�{O��px�&�H�P��A�G���_&{���bm5�z���Y�VC��=�O)<����/\���C5���*�_U�j�������*���3�oT_|����Xu�~��B���F����
����Is'������eB����:Z������%�}���k�o�:��0A�U�^��_8u�����<z2���������"�N7rl{��V/��x��[���j��MJ��B.��F�Gg��n��v�&�H��!!$��a�#��������
�J��-*..�RX�������~!{��>}�e�F�L����O�_����s*�������"�cO{�t_��_qC=G\��R�\����p��=t������[��=��_������P����������q_?q�e�3}��Fh���rXldP8����COW/2(��eJ��%�U=�����-M��le���sQ��Mm3���I������<{�49����	!\.�a��o���]�����Y���o�bR��e�\.w���s��'$'B^�~y*�����2b�����wi�������	z.I���
<��FVfN^N��,�y���o�?����cS���<zb��(��\��"W�<g��6[�j���/�\���^����i�����a^ ���5�?�y�T���/D;
�F�����[�������G�*)*���V������g���G�������j/�n��������y��h�kl��9'/GOG�����	3�ut�������xT�����(w��G� ��D�UUTe�\�J�)%�U=����/�|�o��*�E��8�Mmcij�����cb����t��x�����!����;��Y��������K��^>�r���I�'&�|�<�UOW/'/���~��T��NX0Y�c\�U-
�ZUq9Y9�=�T�-M�-^��r]m]�+�����}�.�:��O=���)%u
��C.��l�2d���+6��8r��u��|$��r��������zP|��Is&�y��
�'&&���NR���������x��v��i����]���s�s�l��o�ZZZ��wg�.)
�`�$����r�������ww{�����~,�XT\�6��S�,�<���c�������u��#Y4;g���s'�FS�w��~�<L��y����0�PW��mgjc��+.)����bjc�f����;�6�
�t-Lm,��@o�l�K�E�x��4@%[�+PT\�2����h��;E�B����'�G�g�k�k�7`���
�
�6v:�xT���4c#�a�.C�+�kJ��oR����(^YI�_���g,RTP��b
?B �_��Zf>�xd����r�r�u��:L;YVV�^v�(%��$���
�
I�x���;��q��i2\.!d����S_����A��GN}KUy.J�a���a��B�������S��5�������R��jij����3a���"!�y���gO	!2\����U��3�O����I��2�I����r�I�9qt��-���M�3��S������,J	�
���WElZ��}o;B�����������'��0w�l����7m�f����jK6,�u�VH`���>!���xB��7���b~V���/������r��TT�X�xM[���V�������-:-3���h��)�9~�	VRV��|q-� �l��f3n��2�z���]�w�:'Z�j�J��m��LB9�X�X��n���1o�����M��y���E����R�W����b��PU_�X\Q�����,.��z���~�U��:�c���dw������S��?�}�}�VN����x����c'���^��I���/�����.�2���\�zq������Z�Z<Lx��~���Vk[��)�
>��7�����yKs��.��2�������g���������*)s��=Q1��o����������\��qS��`��R����S��7l^�xu��O��[�(�8i�BH����/Zs����������{�]�a���yC�[w�&��x�b����u;�7iN�P� �\���`����nk�J�L_�cCFV�6�~9��q��9�<^���M�|g-�{<����h���"c���S(��P�����~�����������=v(%d7�SN_��p�L�2�cN[�Z��xn��L�����	:~����Gwd�}������
����q��M�64j�(�U��
��mZ��zkut���{�G���em��-����t�
=�.�:�	��6�r���uD���&&����+�0-#�q�F�K��W���r�;�;5�$�E�������,�/0���(��(�����Q�zQ.�jOW�!����tW��
`w�P����_G�����c���}����x������+zv�A1�3��d��<I�v��86�q�]���[=�;�=,����\'�Ejk��2k1!��q��'����'���yu�����#+R��	��w���IBH�f-�w��0���b��E)aM�`��W���:0�a�����7��w�?0��9����k��}Uy/�� l65OEYEEY��<�UOG����(u��}�,:2g�6�[�hi�8VT\�����d�g@�oT�2�{���4��?*�@&u��Z�c
e;>~���QVR�zS���������z����g)���<;}0r����0��Rc�q����w����i�o)��w��l�������j/�������gO�e��P�3�m��q��rqI�����^VTPj?$'/�KQ����o��9M*�.���!
����|�m��u����AN����#C�Z�>�JjCBH�������������t��_��,�8f�8�6�k�|�e�Q
/��k��"��4~�Qv��]Fs�k��������Vy���k��_�}#7/��y�5�V))*�;��a�����T����(��`�_��K����_��y�<�/�U����R{���N��=KJK6���w%���zF�fxL������zQ�/v�D��l��`wiF�(J�(YU���8���2�2���;s(�����3'r������}�������] �~ux��bh���!����2I�C]��|��������h��r7"
�>K=�M����e��|����Ez��wou���u��Kq�UU���ww#�;����e�|V�_^��R�2�����?y��e��S_��{k���R��.��5��s&���_9��ef_�}.��e����I}�_u�2�z�aS�|)��49a����^kijiijY��\���r~���d�1�k���v>|x�!������t4���
v:�I�4V�>������@_��WeE�v��y�&
�����T�f��"1�49�{��es�Xu���eh��i��v�m���.�:�fOCO��9�o��m��Z&<O��d������B���q�8��P���#��=��[/B������r�?�������O��������[nBN�����o�/k;��O}�j�����������-*.��dVc�����p8������Q�b��I����My��x��������}�wi6v���M[0��>ad��
�
�.�_:�{��9����2���^/v�9�3�S�T�e��q��t|��de�os�{�"f���"�
�zv�"�����K��E�vj��_�[������2��bwMD��&)6X_�Q"�R/IY5|�����
�TT������;���]RZ�cw�7^T���@����[w��4��������U^���b7���/��%��������XD���T�()-���e��\�Yg.�SSU��!`����G��=$�p0��y���U�����?�y���m����4�a��b��%��5���}���&'���Vf���������I�Bu����6��.�}�P(�����f��� KSK#���q����>�}M��B�N)�����0�!�60l��m�C��J��^���;zH�j���+r��>������������TT\T�������d���B�
�'��Y�BSCs��������>��=Q��������s�\�m�����
g�b��0���u�V
�
c���������gh��?���^]{���x�&��49��������+�
	!G"�����FCM������������9_��Z�XIQi���������(7���|c�F������_qB���_q�������p8
�

�
�<����oTo��B���f��	�	R���0gq`qj���,Z���Q9��\��C�^:W�+%��x�3�:
���y�xD�7��P^N~���<�)<��)���\���MRl|�������K�EV
�*�'"]m���/��no��������������*~�P'���u���9������l�����9QV�T�0m�FKCK(�
����\B���u[�S&�L^�J	;>�}2!�I������kkJ��������G[7o����p�������3��`�Kj	k�P(����s�_��&�I%�%�"��2kq�^:X����*NA^��I+�<u��9>�_�+���{��e?�q$i �$}�2�P��}��h�&
�d�e=,��c��^�Co����_
�,<u�L��g���d����@/��UY�u�s�g���=���r8��<��Qj��q��0����c�f�o�d=C�C���e����7����9�q����	!�o_�>�w���^�(++���2=+�y�:�����V���f���2~���\��-l{�?s�:�P(�|�:�i�h��f���]�}��[^�P�Y�rcCjDU*�7hT�+����~���n1�_�T���SA�F���:EQA�"��fw��;T��T�e�U���r|�0l��=Wo���g��o���S����\�_
}�l������"�y�OI�%R���lRc���f,�+r�U�J�C?����t���=�?vW��
`w�P���S�B�PGKGGK�M��
�
[vour��c�T��r���������K����1IB�����<5���Y���y9|�@(0��R��qIe>~x���m�	!�-M�G����o���b������%�I��R�uK�<K8��H�����^�+������ME6U���*EW[GNV.+'[|aQq��qv���?���jjh��L�(i ��r)��D IDATIe6BS*��f������x<���r�Jv��8����M�~,������/�+BTBB��^�����cN������c>~)�r���A��Rc�Gg�5^>�o���=#$:T�4�oal`4�a�0����E/����iLc����J3=g�{��X+��i>�/'+K��pTUY�w��R^iQq�����=��������"'''y��B
���h�|c��-+61K����_��|=�z�r��S_$>O
�u����	�d���Hm�r'E��b}�W�������(!��5L�5����_z=:[E���g�����X�07}Bf-�[ZZzh���F���y�vRw!5z+;�H�&��w6Ql��4cb�����$]d���R(J=�~����+Y_T��N��~�Ba_��n�\'�y�SIYY>�/�7f���<����Yr��Ss�����2�2��T�*���������B!�[��oR��
e��{O&�L�Z��d).)...�b�s������|�@ �����8\��b���@r	�����N��xsV��}�xLS]��.\���C5���*�_U��#�
T���hc~��eOf����GNq	�u�i�&�Ue�z&?a�H��Ie<J|LS(�Y�e�M����[3��
�dee)�{E��i�i�Y�6�����R�-����g7�K=}UJ����7��,\��}f����~Y�,3'�@W���F�Gg��n��v��7k��iKv�Hz����&���t�t�%m[�!�|*�Lil��I�����s�����!zo����%>O�45'��Ry�������{����
���d�oUy6�o�<��h��wo�~�f������q�����#'�D�=1r������D,b���6�or��Km[����w�����1����~��SX����:5U%DY��s���2r����S^�\����-��9���O������>�y���2���^�8���F	�f��f'�"�&O�G������o�
�$�Qu�]�E���_
���|�@�����q�c ���k�3�2n������~=�J�����{t�����3�2N����=�1�]|�k�
�q9�Js�uI�H��������#�\�8B��2����t�����
7m��^�}��#�?MN��S_�F�
�c/�,rQJX�=H�����-�7�pe
>|*(��w�=x��mK�r7[����\�z�aS���0�q��5O��\�}c�/3��������$
d��2�c
};����q�ffN��g�#��5�B+�za��m_�\n	Y�RO_�r��o
���
�����^C]���s��Y�3�[�-X��K�v�hX����wsW.���~Vn�������3i��!d���oWbc�����^�~9q���{����{�;}����G�
?�%*�-���>[V��''/��w���<��cX�`�m��4�T�����=MNX�qEfvf�-4n�h�l�
;7%�|N/<%6���o�����t�����}��^=R���������X(��<�g������0g}����k�9Y�2�D��*SB�]kT�9���!�W�^�Z:k�~mP�~�v�E%4����LI�J:~@UYE��j���K�	!�@(��E9�X\I��&)lX\��#�R/�EVM�*+r"�T�������J@U���5��y���N�Bv�����k��9�vn�������ce3k�LI�����g���G�������j/�n������@W?h�����G�j�kx�r��A���3�3mAUEU��-���k�B5U��>^������v��O7Ej1X����&{0$:���|��E�J�Jw~���/��3���}z��2�z����A��6�;���k���cg��T��om;w�l4�����T}L�l��Y��j�||Wg�dy��b������=v$���@/��U)�bO���WA^A�DFF������3\�W0�e���+6��8r��u�Y���
�����y0`�����<uU��]z���a�=i��Q��=G�.��BKC����,��L��.�����y��p��P�n�Tdw#K�L_�z����L[�9�u�h��	�����~*��a����t=��GL3���wz������"��(*(R
/)6��.���4���%�N�	!�;v�����1��k��h���@���B��m�;Yv��3	�0gw����k��+����jihRB���=G��/��0t���^��?�z����vl���a}�����2S�n�\��u��n��u;6BN�o��%�/0���(��(�����Q�zQ.�j�TY�Q�|���]�-W��@��8111�v����4
�P��m�a���r�Y�\��2{��'�?��
.\��"zi.@ms����V�i�I��jiiu���yV�P(�A(B��\n\\�P���	�.�5���,&����-����ejc����0�PW=�|�fvT�y�G=T�P��B
ik�S���]�u�~�<��^���bOO7C##�-;�N-**=f������l$2*������4CC��N���;��_�N�����A����M��s�.RPP�'����������������0~�dYY�
����J�z��o�x������{��+2<<"$=���������	��\n����b�����B|��&���NI��^����
�������1y��q���!;vl���mB��x����\.W]]������fm-���$B��500�����I���u~��I(=z������tuu
;�S����W`�������
��\Mjm����L�1!,4�a���B�/0��6P����m��i��������������mhdT�:��>~����B���+��6�X������'�V�^������-��|*�1cB�>�/Z����v��#G�O�4��t�����N�n�o�������O�p�'L�`	��)%���MM�v���,''��p���m�7/_����}RR�J�%

���'�pR�q�@�@����k�����}���*�
@� lj!��&N����.22|��-�wt���I��w�>m��KMM����h���}G���������+��i��y��>>�p9��3�����)v�2e��2W��^�B�CI��P�o�p5XC]s������?��z�G����]uu{�L$e;�a���OB������w����������"��t��[L�W�Ap�I�
.��'Ou���Y����[v�����G,a�5��������������z������m�\66����f5M�R��R�����s�Ned�o�
��/v�\���M-������I�&]�t[�t�����_%E%}}B��q=B��3��������IUU�����`��K�g�I�#��A���L� ������+4�Xt����w���n���q8B��e^jj�����n������l�t�*%%���w���������
L(_`(IP�L�F��7�FE����B���:���x�:���S^��lD(&&&8�t-17���:!�O9�����[��u�������j���]\�0[�$��g�{�b��f�Z��|���[S���H	��)=������z�NNN���v0x��Q���N�%^��wo�i�����������������Ua/WGl�aS�
0x��Y���e���p9?����������o��u�������pI�1�cY�%�����zEF�<�o��m�����%.��%++�2j!DA^!6���������!�m����wwO��G"�����>�=	�G�VT[J������jU�}�--�w����������b

��c�v�8w����Fii����X�=#3#&�������c��?xp��CAR�
:�a��x�=;��;;88phEJXnM��x<���:��8x�����;�uE����0q�����R����kc�O���ru� l6����!!$;;���������ff��F?���}����:.Y�5c�\���E������L��.s���w,�PE��[�q"d�gK��\.�U�6..c##��$�cl\��q!DCC���}��B����������'�H�CO�� �C�*!���+��CEuu�V��hhj	�BB����K(�i��y�c��&/SS"��=='���9p�RL���-[�z�<y�
o99���f�K(�����u��h�F����f���?sR(����d��M~
�|�~����$$�����9��)v�\�����}!���K9}&*&�����J{����k�O�������l������T�g1��e����k������������$'?[���j�f�d��^�~u������]PVVt�6�ls���bE�?O�#��
T�p�����[��x�r�+++++((�����9v���{N^^AEY�u������E5j�����R��t<��x��m��BZ�6uu�+�o���RjZ�z5i����<����SZZ��jIbb���G�6i����^�7����z�������ru� l���rex|��cIq���L�k���#������O����s8\MM���)?�P��������ek��C�T�Wj8���,u������u7���P|IA�����]U_3�K�����&O����Vnj�C���/0�JT)S���E�O�>~�"y�Jo�cIIqqq�����M~���eV�p8m����og��,����\�7��>}�����7PSU�HL��]��w�P������TCf��$>_ D��x\�^���RjJ���_W����Gk&)�Y���3����~�e~^^����444�W'J-��+�:v�����u�������#6a���.�4����D��}�'�W8����j��TKK�����R�����\B�p���#�\��=��� ++�+�������
��YH*3���F�%iio7l�J�w��]�����>���*))+))Q����+�=	�C��������'�����3[�6=+Z������#�����*w;���Y����Y�N]�������7l�&����q�W,^�l5��;���\���KXx�_W��))�CB������$����Go��c��-SS_��������^Sz�6nZ+//��q�;w���8>s�<f���A��<�t�+#S����!�r�����\��b<~���n@��-����r���8��wag�|�P�N]�6m���46���	SE����ii����"��������G������C&&
[�6}����C����2�(5�E���c��A�C)s��i�V�:u�^Y�Y��
���_������Z��������wo�.[��������0����J�CI�� e�7�X`���w��kJ��G�>�![�������o Z�����r�����������C��n�P�^�%K|zX������������o�����nn��G{0�Q���^����d�Wnn���n�>�'xN��PNN�RSJ�����o��V��d���5����<���?�i�h�JJJ�_�U�=(�YY��n���^����a���.���(,���c�������F�7J������^8���b��m�� �v8��O�����}�����^�l�L�Y��R,�e�)e�Zu�^�fM���UMM��}�}9pp��u+444mm&O��-
B�CI������i��NRrm���������a`5m����ZZZ��wg��&
�
���.��G��y�����L��i������d	!jE����j5��P�a������(*.v��fl`������G"�n��E|�y��Lt���v(*�����!�G���1�������B�2��j��u�>n��KO����$
��<y>*=3]C]c`��'�T�W�H�+�U����2~����k6n����SRwv:�xT���4c#�a�.C�+R<v�j�
B��)�O�=%��p����V���?]GK�I=s����������u�c?s�t9Y9BH��B� �+����VSQ�ce�5m����+�����G�e���;�v��,�}����	gG76i\w[����s�����}]}��S�N�����X����N���\���������r��
���>|4om��w�h���<��PTd���}��QUYE��T�����-^����������(�+N=��D�%iop�����{�i����������rL�G�2������F����/Zs�������������vm^������a�#��K�5��[�VG��l����d�8o��R/�u��]����=p�r�J���%�.�H�L[�eu�����B||�<Nz�i��F
��JY�a��M+�Vo%�D��Z�m��	3{u�y�����eddG
��4W��=Q1��o����������\��qS+�y�y������AQA����
j^��ut���9�i�������'}��AGKGYI��C!u�������qNcO��-L}����c@_B����m�~7��d��(I�}Q��U�V-\��Kf�V���'I�2���������7.G��J�H�5@����86�q�]���[=�;�=,u6�]��l����QRT2�3 ��7�G��=�]FZ=C�}�A�;vc�,�U��� �������yp�y��vm-	!�����I~���������L=��������A���8���\u���E�L�w��0�QecUVF6��X�����
r2�a}���
j���'I��<�9�i��-������w��I�P�����Kg'>O���b7�u�(.�[��P�wQ\R�l������<cv��Y����e�"�I�}Q�\��0��|��������h��Uf�U�����\�y���+�-��+�P(|�,a�p7�����gN�.�js�p��6���.!���~)|��<|�0QR��}�������Q��?b�z�2�n���s�v�	!��9�����>U��Gg�����e�1��?{;� �'���l��y���wo��0�������o������G����j���u�(�n�2\����5�[��S^=?{�K���>�Ka~�\�R/���a�C�m
>`������d�
�.v0o��c7�����0�c��$����tc�fK���Wy��2w��`�Uf�U��i��'�u�um{�����_
�K�5�5�_�1����9���QRZ����*�U�m�B��7Jy���oiji�o���G����+454�:�}������c�g�dBt�Ek2?g�f�0�4�a�������Z�v1�yP�A�����F�.��}%�b�6����������3�!�<
��_eU������w�����o���R-o���|E?7i�$;/���`���5��'IOb�^8!)�P(����s�_�`��D����Nt�����w6l��������qBi����CG~���R��T�0m�FKCK(B*��].@� lj�����\<���Ky�6�{3o�U�W$�0��9���
��F����Y������i�������?=�z�p�r�r.W��{{�����4W����s<�h����S�������5~�
*+)[��x�����3��E?C����~3�����^����;nBzv����S+��Q���<�'++[�� �0�zVUV)w�R^���%O�%�y�y�fL�`u�$	�B--�6�[+�+l����q8����Fo�k�������T��r�������}��9y	}�-�a�����z��5~:�������u�u�d�2�3	!��w4�p���U[���e460�0t�������oZ��4FW[���1���O���g�� IDAT��a��P��)c'[�1'���4u9f�^?vS�<�����Q�/_��0���S
�d����mP������v��S��i�%_��B��}��U;�^������-���qU�1!9������o��O�/�|*��.%>N~�����j�����wfdeZ
����>!����Y1�m����ce��(I��HJ
�}F��?�YVV����������K];tQSQ�Hq8�Vfw��Z����X�����WU.@� lj'Ue�z&�����]��������+�
5����1II/�<�N�}�+Z�m�6��O��u��
t�o��+J��(^W[����?�@ �.N��l���x��]}G��8��8�J�E�i7 ��mP���M����6��	���w�<s)�2���X�~-,zQqq����/�|>�/������^�m��;73s�N_8y|��q��C��W�P������X�.��2����t�����
7m���78����[�o���|*(�TP��@)I������p��=t������[��=��_������R[�fz����O�miZ�f��/g���SQ�Egde�8{�c�{5���6�	!l�jMr�|��d���'��>MN�r�wB��iX����wsW.���~Vn�������3i�����0����	������_��9�4p����hfN���4�?A]��}��#�?MN��S_�F�
�c_�X���K�L���/|��aC���Q�6����a
�����dd��}������@��'km�GMu}��B����n��]������e����.��Y9�FF�3�Ftb���%�BNV�@�@������+Z��C���������v��D�%i��9�vn�������ce3k�L�Uf�U����`dfg���W|_��[�,X~8���7X��S���U�m�B�@��4�X��c��1��TUT:Zt<�XCM�rh�����,���SWU����j/y9yB����_>w���������`c?�}��f',�,�1.����fj�i�T���xe�f�j����?}��������z�5�v!�
��8111�v����4
�P�����Mk������h
�!����=��j`5m<��Y--����3O�
��?�BB����������oT��i��k�O_8���cQq1�m�a���Q�Gz�xl�����>���B��
�#Y4;g���s'�FS�
�;��P��B
ik�S���]�u�~�<��^���bOO7C##�-;���z�������x%%e��}��]����b;������"�d��9����K���F������b�%B�������Def���k��
�:e���=��?p`OL����]]}����O���%������B<{C�Faa��%Wp�����������&xN��s�.=���NGG���q��i\.�^x����'B���
�F:��\�����"�a���>�l��{L=z�1�D��[��y��1�51�)��UW��0o7f��Y[f5&���r
�v�>i�tmm��Y�C%�+;��;��;��q��r��^��w�����a��%J��F��6�L�3�������������s�
����
>��1�O���-���]�v��#�'M�^��P|�����l��^�99�2��
��������h���Aa��W,_��i���/||~�r�3g���:p`��s�|��7i��������s����B��5z�655��\��8�b����[\�qm��::z;v��<w������/_mi�>))a���q�����r�����+����x�����K55������\�"
6��������������2�c�����d����O����RSS�v�.Z<7h�����!���a)C�c�G�*����R(��H�|]e`����<j����(�1���x/�t�T�����y�	!��5>���Z��P|��A[[GIIY�
�������2���H�BUU�_�Wt�����o������$���<y��S�f�ZBZ4o��S�'O1I����;w*##}�o�����p~����UOB���N�GD�T��/����~5v�G[��ll����M�����S��<xx�>��
��7,<X�l�\�"
6��EYY�A�
L�t��d���[ES�J�J���c�z���f�����I�0��K����2�V��[���c��'���ihh:;��v��p!K�y��������u#//�����%���������+���4h`B���h)�����_�F��7�FE����B��/�t���v�:l���eUU5�~�..c�<S�"�����C~~�B����IZZ���1J�������|�/Oy���urra~����o��u����D��r��g�{�b��f�Z��|���[S��,S�����;�.�Z&��b�����O��?��{p��Y����ef_���D(&&&8�t-17���:!$E�\�#
6��`�������000,��� ���a*+iH��q�FX����Xf7�V��[������m����������^�dee]F�!�(�+���_��{��9>������i``x�H���������-���q�n��_��+#3#&�������c��?xp��CA,�C���>|x?�q���������v��AQ��}����;w�Vn���7���q�R�3�:,5���C8����gwwg�A-�Nhh�����M�o�ee�+2*���TBHRR������k��>�?1)a��iR/IaaaII������1]�Y�;wJCC������OU��uD��[B����,OMM	��offih��=���!���a)C�c��[A7o�>T�5�V��E�1����=��m��������p&����stB�����h��,�"##cdd����o�.S/z���,����
�m��e6X��������pO��U��_���~n��INnvXX0����'��\(�����������;��o�����:�s9r�����}G[�l��y���rr��'��PZZrd�����%�������ss.''��Y�~QQ'�c?����d��M~
��'�[x���5Z�j���%
	!�by���
��.�R���{�/!������s<��+����"�e?^���(a\�V��X�c��+��=;�|),����3���V���d����{����{���wAYY��	�g�3+���r�E��oiC��F��������������}l��qnn���~�&�������!���,^�\EE��5�B������N������<\ Pr9<�cr�������M]]��
����q3����WO�o�����n�������bUU�����������>��$&&��{�i�feR�-�$���


yy�vv�]�t#������WPQV��\�}!l�\\���},).���)w���w��y�o���'O�p���Z���uZ�C=�+>��x<��X�cY���e�ws+��||����������z�)))M�4������r�2���_��WR����kR�~k��~�L��]��w�P�����TC��$���SfV�������h��_W����Gk�c��DCC#YY����x��b�7����������z�&?33��C��truw����ee>!�����r��[���@ �5���r��.W�\��������Bv�n�u���O}�����
�W~����:�$���/���r�?�����
�^Rq8�6m������sd����cff.U��#�
H������&����k=��(qg�D�nm����|TQQ�_���l�P(iH��q�GX����X�l�2�R��k�Y���v�����z��z�5l�8!���c^^������%�]��s��hy��!�6�:|�oqq1��O����g6*((r�\''�������.cSR���;���v�n��[�b��e�����l��V^^�q��w��q|��y����MOE���9}&�����}G���8������C&&
[�6}����C����r�\J.B���}x����;6o�25����0[[{��<~���n@�z������c�--��
�������:��{�����O�:���)�T@����]�����h7�_�,hkj��S����q���
�US.��M
��s>~�s��M�6KL|{~����������w��i�Q��������3�	eH��q�GX����Xf7�V�Y��i+u�:T���,yy�Gf�����g��V���}��{�t��v�,��l�y���T��WR���X�2�y,����;��5%����M�n��-[�{X�������g���'B��5���G��`�������o��V��d���5t�!DNNN_�@����*��-�>m�������s�rttt{���2y��\sf/TSU[��+77[GG�O��<��&++S��;���<hXfF����>}���u�����7?US�<���?�i�(UII��+��������{���!G}�n�W���%>����\�"
6��3v�Ga�g�[�����7z��QbO���p>��yee�6mvYX��I�E��D����������n��ru�^�f��;�c~������o���/�]�n��������I���A(�^I�B�2�����i��NRrm���������a`5m����ZZZ��wg^�*
�
���.��G��y�����L��i������d	!/<�����tR�4@CUjQad���d|��%	��dj�R�����/O ��%	jL�@�5��������J���������P��m���xvT�h�	��:�����}v�~e�C1�����������������(�q#�Y�p8��3K8�xt�7�L�PM�c�|B���oJx�ia����X]!�s�����7�
�BB��3�k�mtu���?�������[f���2\�{G�!m�����?�����|B�%�&{�Re��p�0pigh�����8�^z�����].�M�9>����
!D f��H��}�Mna)���Z������b�������o�i?e9��=�n���*��������W_���es�1���2����C�e0�;6P��l:�����/u�h��V���\���}�5��q��@�Bl�(`W���QQQ��^�����bQT���`W��"z�Hy^���C�@�}?�Hvv&�3��Y2���zs���]����#O��(����h��f<�(TS
�$���yB����/���S*!��\kd{CS
��o�k�2��~.*���9���B��$�}���\�)�5���_%9�����;�(�%��������a0Do��5��/E94T�Y�����z#ZR�/�%��N&�lVA1_�dd{C�V�����O/l������@H���D_`/�Yv�K�><K�m�����QOp�a!dL���u��DH/l����_�P���W�J����ZoQ��wi��4�95L/(y�9�>�6[�f����"=k��f�����Iy��TV:7�����.�&rIQ��A�����i��|f1f��3����n�s�5!����J���'.�����^d__��Zu-���������K�����k).wj�����so	!�-t�������1��T}Nw3�@x��s��~o��h:�Dl������Y���n�"���R���������BBHN�T�v���ab?B�����C5�V���SC�F|���e�Ps�[�������T�s������>��>�����Lc]��y\^r�'�����o���o����V����]���,lo}U���)���&��,,),�W����J����&�m�+Z�����w�c6!$%��^|������i(~��N�������={s}����o���<���B_Eb%87�>�<�N\!���k#��z��.MRm� �.W�qE��y�z�]!$%/����w+}��q���.�a���Q�J)y����.!d{�&���\��������I���2b�X�� !im�z�E���<BH����W>4��~����F����?L�r������������hF+��F�X]~��^o����>d���E���!i�%�^J��$7�����1�[P���U�9��~|��^�gt�Wj��l����~�]�O<!$:� ��dHk}yY����es%��R'�k+�N
��C0�WV�'\�ER�dle�Z�:Jr2�����'>�#E9e����X[����*e���J
z�,�FX��\��`����	('��#����D��T}��x����,^��;�����������)G)6����b{5��_$VK�!^!I�����HL�����*5.�AH3���gI�%/��[q��MJ��%uD�aSk���(+�H[1�U�(�zl���L�u\�7s���2�[7?��C�F[I�DC~����\����4�5PcS���Ao��*��D�����@k=K}���i����t�	���r~���G�����m��"Wj�V<��,�	Gi��O���E��D�|����o[�q��d�{�p�_S��=���w+��������U��r*N��� {�I��������f�('#��i��q����rJM�Rfu3�j������f�����(2��a���+����S�i*bI�=�(���T��p'.���������oM9J�M�5����&���`�]�R�R�d�X��"�c�5�����O�/��a*�er��\�sIQ��A��3M�I�L^$�%�r���	!�����a�S�������}�	�I��G�U�#����7�H��Q��[S������2���({tp.Op�y�����P���e�3��+�u����e�h��l.-E9!���}����*�F;�~������r>~��J^a^�������0���_w#�&fTf����3��(�y���fj�q��k
:����%\�Nkc�:��)��8�2��@
��I�L�����sV�px;CGs-�����4�V\���'�����J�~������aV%|alz�������I���]����-�E�d�Qo
k0 l6����u]���0������U�>B�xB��BZ���nJ��}���o��\���(}Uv;���5�Y�J]u5��h������-[���JbG�������o�_$�Uu��2�k����wi4�(��B�bU_6���>��d�uk�2)���Dt"��e��5T�������<�kk�.����!�>�Cz���,�� �,ul��Z\E_s3�(,�(,�N)(�	�u��*���E�BB��c�s�f%�����E���1������<B����cO��t6��-�
��s/���a�Y�|.j�z��,�$���vY��-,�sy-E�K�����	!.M��y��b^����a��.c��/B!��V":>g���6!$<&=�k���������I����S��Nuo����A�����B����h
EY"���:�f���h�U�g����]��}��v�Y��w������j�zj����f��&����rsQ���,'���b���N�Y3���Q���o@������gc�G�s���,G�-�P[Q�����tm�!zk������	�����(�������7'��hk�Qa�����P�AH�����E�x��A_ !��dl�mn��r�U�/������QX���-�lL���J��|.�AH�F���}�!MRm��e��[�����N�oe���,���H��q��\�!l�F~1�Kv���"���	��m�P��S�{�^(�Dl���?�BKl~7*%��,'�YX��_���{� IDATZ��[�f�$��&S������&�kN��[��>=�"�����1�'���J>������g�I�~{��\�MJ������*)�G#M�
������R��,��ZVz�����[;���O��YLEYEY&C���^3�M�Z�+G%��x9�f��Y�5�(�q~�����*��6��j��q"!$*9�y��AG_P�'��:��Mr������am
z4��S�kg�6�������9�r$��loh��2�b�@@T�,6K�-C%����l�����d0h)��������[	�*r���1V�R�]��@CQ�Xd����lA�]�������'��-t�4��S���B�����'I5�KbD�a��8�(�BOi�}����%jX($�������6�6T�Q�k��<������wi�!9�:���v]%B���B/��W)����U�3Pe��������G+���b��|T���WS�c8*lmE9�����i��/��{���������Z����T7��$1���/�8���7�(���o��������o4�L4��86�L�*,oJ�;�������h��(z{wJ;B�����qY�]����/�\ko|oc������\rw���3/S!%|aJ��_��[�����"��n�ZJr��7�g��{s��
$���SW`��RT���������7������5�V�K/,��6c�����Yn%��J�Wa�qk�"'�R0&�M��W;�$�f��r�W�����W?ic0������e����GrIQ��A��/����L�59��SP��%w���9E<B���������k*��q�w?f-�P�B�?Lh��0�2-�X[I6�m����w��3���[�������Po��$v4oR�����p^�dL�d��aBE��:7�vn�]P��J��,!Ob������'�����.�]{��=�3}�n
4��d|���5_�������q���;�m��k^���3:����P����_��1�s/�Cm��Z��+��i�VC�?yzACC���F B�B���PH�^0������W��*�%��;�����
|�4��io����\����Z�g��/����J�WaO�Ro�}���b#p�n�y�O�Z�)U�M��v\���L�����!+�4�(y��]+=���Jy�4���&�����/@]e�������-����>���
�.����X�Y����������?~����sE����S��_���@��9���������|>��=��^L�L���u��<n��U�rh9ut��
�K���:�{���Y�gx���D�D(�<x�RHRJ����[O�)#'��������A!���	���n�����a~����%~��5��"{v�1w�y6��~h��6[�f����>����f������A'��:���h�������g`e6O�\RD l65�xH��mk�.�5n��A�!a7��=��%^C]�������������c����!��0�z�z�mm&
�������[E#}R��]z@�)�f�����zUU�����D������"##��y��Z�����6j��r���0o�d�"j��k=x� pWG[���Z$!��{7�.�Q�
+�V�������f��()w���=~���J��4�!E<�Wo��Z9����s_�a��
�#���<������e��X6m^��N����������#/531�������~h�����cG�?"��{:WW���(ec��t���\Q��H�����<r941%���?~�X����E��&�hJ�_M��-(,x�*r�����i}��]G����������
�����7��dN6��������l�b����%r�r����wkn~����h���cg�����q�F��?�]����9v:}�4Ig����[�j�������F�]5_^N~����������G:vuX<}AFV���K����4|}��lam� �~��(�EkN\0��i��u���u;��������e��y�h�k8t���\�E l65��c���B>|�0~�$�U��oDQSU#�\��1s��!��W�]����j������K�����u���%<^���u;7N^8����?��hF�$eD�rhF=�AJ��'�j?�9|U��,5�K	
;gmau���)#&1��H��y�����~t�Aj���;��`0*_QK7�x�����kL�M�>��_�p��%[�o����&6��g���9e#�z��������R���M��x������t��4��GZ�vHq���j��I�KP�^Y�F)�4�s4�AS�4�$�>��x����l����w�]�����e;�N8k�)K�`h�J������S��>��U��zy�22;'K<�e�+���oLi����u�����(�FNn������bE+<�z~���aC�\
-TUVY6kq���!�m[���o%H����/_��T��]z�{z������`Mu��&�%��8#����~h��6[�f��Ev619i��~?���c�]�9vs ���p��xz�L��)�rIQ��A��%E%%E%Bu�U-mC=Q�����Z����5���<|�X���RA^���!��B������(��wE3��$Ui@�)�f�����z���s�������{�����c=�xB}�-1�)����qp���m�
�d��x2������	!�-[-��4��{����g��T����>h5��-�.�������l��S����oEE[�o����1��K!O��gdBs��
)���z	!'��<w�s�g-
�^n|�������2y�e3���&$'��MDU�~���%$���MHN��y�y����Y����f�(�����}|/#3��e�s�)�+���tG�R�2�X_n�JWR���������w���tf_P������Oq�}���{G���K����x#�Fvn����D��N���A�^)�Q��y��9����^�i�.^�����k������<"_ER)N8�kJ�8�9%�&��x�P��aA�Ogde�w��e�F4��7%���O�^=&��%�<}Y*���q����wo��7~����SFN��9y9Y9�L�y�VK]��co/�A�_Ls���-]2cQ��Dy�{R/�|���"��3a��i�������$~�f)!��>hk�z��-�"�++��tw��B����-����/����
�7-��P�-��B��w���y��X7�:�/��NS�\RG l6��[��7�����w�Zkkk�rWf2��&��'��B�T��D�rhF��)���t���2���
�|��e����w���l�y���������y��/�6����nC�a7���S���[�M��8����m+6�ln�>f���,��c!�����G��n;T����gN�
���!D_W��9��K��(JU�g��%����e����lm�*���i�g���f��*�����0�����YK�~UQ��_f&��="���y=q��u����}Br���#!��Ac�~��F(��-��x���)��������gx����Q�#��K�#�)�Z�D��Q����^�������ag�M�C�y�b�G�s;[B����Wo_;���G?�������l���N$��J��R�Q��t����d����M��x��pB��I}�YGQ���������;�1���[h�k��)k�)j��n:��g��9��q�5p���}����r����]�rh��x�9Y}���[���K�M{7�<(J�;���e+�6����o��}��f�;��co�������ob�'����&�&��zQEYe��Q^�w����%���-����/����5�����XB��Bn1W]U���0;������j�%����K���
��ede�B-m�k�}�������Z_���7�B�T��D�r$�ze)��^)wF8q���*s��>���%7{WB�k����q�*��i��Oa��*q��h����j�n���|����:��3Q���@O���-[1�L���|<��<D%�������i�&l9��~���ddd���QW�Ut�����3}�B��:����������U3��K��j�3�|���
�
���+��������231MJIwF��;#!�)���e0l96[�]��#}Cw�>�uU�����c�%��tG)R�2
����6���U���d2���_�v��WB��x��^�p�OM����
�25�''+������QOC�^)�Q������U��^����I]:t9y���/���7���_�������&���"mM������bfl������������M��r�����O�q��M5�����b��,k�����[�m��_�~Zf��3����F���R���)��G�rp{���Z��|>`/�i�(B�p����!'v��*�7�P(l��9������Oq'���S��)wk_��E).)>t��_��V�����j��i�i�BR�J�~�
T[N����V8s)����|>��Wbg�m���N?�h�I*�f@�)G��G3HU��m����[a��g�������k�@����n?�SPX@��s�NL������]%���{��:���ws���?�8IC=�C��'�$=�|x��m�i#���[2c!$����G��>���Y,�����$����������*����
�?��,Z�U������������O^��/~$FTu����i	�$5=���{C�{��s%�� /?���i%����g�W����;8H��4h�W����3�������#{n����s�����������U��p��-��=�1LB����G�1_�������^���r��������gx?Y��y���]z�?��'��ue0�"��]�>�MY�M	P�~���CA��c���ban�3`���[�w�����
3�3y<!������RE�kihiih5k��-���{��k��7�X�id��Jx%�V��z}l��Ff
E��j��nH�ob���N���~*��5�����yQTT����2E))(���Yn=]�_�_�v�-�V���~$�\�mM-Y�ljz���".WtO?�n��O`2��j�R#�� 4I�
��x4b��)5��;HI���
*��$hn^n��g>b����9����B������r��~+�v���^��c�S;����}|o�����'D7������}���/�.Z���c������������S�����4���e�����JwX�|V�z	!���O�BHa������[�a�:j	�~�TQ��_�z�r�r��?�������G���a2�
�%6q��"��%���fU�Z�D�R����q
�9���8:�N�l�/����������v�E�����������H��/X�h)�#���4!*����<��FeF��MRVT��d}	����UQR��g���y�z�Y�GQ�P(�@�]��h��iJ�����*[�{N���@ z���?0s�W�������QQV-\�a��M�n6]����1}���������W���s��\.��O���Z[X�����k������b��|���x<���32�2N�:���.����Z\��G(R�@}����G}�UQ��B�-�M4�E�r�Z���U��+�^�E�O^D��t��<~����R|>��r�O����b0V�,���>���Z����c=wkP�>!DYQ�������&��W1�+[6��)��Q�f��L�+KME����I_���$����~ ��U%������Y:kq�V��%�^?�k����T��.}�p�F�v���G����U���
��������}��%��4��%���B�!f&fQo�D+gde(*(*�+B8:�/�_Dgt1��Z[XI�����4�Q�x�d��32y�^����>'|��������:=+c��SNp���/�NT���d2��\�}������j����	320��y��%�@���t�A�&z��
�_Rw=B��^S���w���}v�k|��7/��XD]s�>��[Y�Q����41Os:G6������������;us�e�
��+���������(���x BS�rS��:���".��[a��B>����!.����������?�=���I�r(wF���=�F���7����~Jz��+��4����������A��O_m�`� �f������������� ���c�zv�!��������|�r���2L��������w�������~������Gw��:����O�+��[Xk-H�_�o^67�(��r��w�Oh���������/�����SC�$F l6���#'�~�t����Qw������-�Qs`2��&��Qb9��z�����W-�4������@���J.^����s��"zX��������k+��GL�l�b��9���I����}MJ��d����S3�^��^�������AcB���>�o�������>~�8j��M{�R��<w����W��~w�6��Q�4GQ���|<W�z}<��
;w���"n����k����*���t��ykv�����~��;Q��!��������t �th�������m�s���b�P(��y�k� E�^R�>8HW������D���4�+]mT����]���Cm
��[�664j���h9����b	!I�I��(+*��VM�8��K��B�P�^YQ���<��\Ea#�zi��MJNKY�a���O�3��n\����3`H%���heDU
Q4%@�R���4��}����.	!;Wo�������T�Uf,����������a���R�C��M����s�����i��y�fS�n��d9:�j�J�2L�h���S�T���X���������n��I��'�����v�&�Dy�'�B8������[<������ �A��:4�C�����4������]I?B��������:�j�c����������.�a���)�X���v���;�N������~��)���&�����Q�f����I���$���l���{����%222N�����9��1/�d�_����+6�Z��J)6C�-h���f,���������}����R�t��PT�m����.�PSw��<y�*�p�aI�I��Mb0���g���pt�����s%����{~a����R�8:�n��]��~������l��=���
oDQ��_���&fWo_�mkC�i�a����M���.}�D?G�~�o�?B�����Z�����t��4�K������������N�����}��zw_g��G��?S����f-Y�mM��=��,��HOW��J�������h�U������rf_P��������>�iN�hbCb��������oRJ���s��-������o��z�Y�G�J����M	�b���YY:V�|���a�QM� �-�,����&��z��)��G����vp��9���q~5O�^0�u���������accC��L(��
��������`�=�:�����b��s	!����oD\�,��P��8L��U/�?�����<����*7|��X��<y�5�S���U�u�~�<V]�.�;b������
;D�|~@��C������G����*E94O��u���	������^��q��/"����6m����a�!�N�LJJ��������@j�P(<z����!))I��j��.��N��cB����� ��LLO�<������S�II	ZZ�NN��F�g2�������ZP���WQ��D.)"
6�����v������QoO���m����	!���bb�0�LUU5+��C���hnE�F%B�L&��������45����D3�H�EEE�������!a?��i
������Wd��	G�<Z��YM���?p`OX����tmm]G�����X,�$���U IDAT�	�}�7d�/������G���=-=MO__<�����N�X�he�F�����b�b--�6m�U����-vl�+Z"++G�����8qd��s�,���X�r��#�G��@�a7n\��u���+[4���z�l�uu����	!���:�x��
�}��t�_Ls�������hf��ba��z}����[�/Z��������K��g�������E�_����m<
����\�E l6?�����Q���������~��Mg*���u���</>>n���s�N��w�O�M4��t�s����\%%���4�ls��]��J�s������������3&0��Q#��'@M��^�[MU}����#�s�����\��ik���������S�+���)�Fnn������b��sg�54�fL�K1448#���f�N�<��w��!���g���
��LWVV�k���;QI�����}KH]LL�x�lrr���~��/�?
��`�L�������}�+f�\4�U�-(������j�%]D�a��(**���o�q��Y��oM�*�+��r!���3'%%%������f��"�_�zq��s����?��i
���Rw����c��g�������=��`B,�������������k�V�/SPPHLL�V����DE�j��C���	!���k�1*��M�	S���v��������B^<�,�����gz��.]:'E94rss���g���VCC���w�����6>~��e��~~[n�����b����s�T��	�������EI�-�CBN&����I-����?x����)3KmOaa�����^(*����+$&|�&0�s��Wm��t_����\RG l6����fMNII�p�J%Q-�`2���T��#E�sW�Z:o������/WT`e��jTw�+88���}k�l���~�.f���,�s�B[�~i��y�O��������@����#�����K���q���11o6l��������D�@�5�Yw7��j�\��]�CN}�Oy�6���K���R�C�����d�����eW�^}w�m>z� ����vQYEe�&��C�<���!�
+,,,.����]�������g���JJ�
�E��e���]��_0{��i�.�Km��Z��vv=����G����5j��\4�U�-(��W�2�[]���(@��qz=BHZZj����q�voo��Z����������o�.k�V��u���\Q��+%�?�X����Y��K��������n�d2�4i��9488�Jb0���}!jj��V����&���������nj��������p�N�[��tv����NH�5��[���qS��2������6l����gH������Wo�65����v�du�F�P��Ys�����������#�Tf�TU��4i���!
	!����e>>#���"#�l������u�%���>c.�Q�r4�P�{���gN�_����D<��\4�U�����To����a?�[�7�����s�C��/�x|��s�nsf/��j�f��d�GGG]�q��F�������v����[a���.���}�l�j����~������{���� �%�n���9E^^�2��?r���ka�����7y�>v��y���1�'�'@M�=�~���V�\_RR��r�����������5�����x,KGGGWGW�djj���N}�Z��)**�����GG���;B��/������B�B��������yS6��}��>���&'���(�u�lWj�JJJ�.��w����J�(WE�U����/"�z�1�\(�)���Do���,�L�k&&&BDCF��c�L`0���JJV#�;�T)�y<���K��Y$E����%H?V���5T �/���}������4����=�1�'�]nj����������}�4onIi����s��][��]�$���J���%��~���{��^A~~����B���?�:w�*ui���)����e���/���[��}�.FOO���h�����B!����O���mE�`0�5k��y���+U���'-ZX2�L�P����/�������[x���6m�+++�/��x�5#33���cjj�e���\4�U��������z�=�D����FRR������u�~�#�����M-444��JJJFF&Zu��;��p�7o^��x�<�mq1���::u]�n����}���-�$��e����Z���u��M�������^�zf�1Q����

�


4Y�������n2���&	jB�����R�~��|>_@�����g2���)��m����W�t�^���LO�!R�C�p�~���s-\N=�Nd���rrrff
�<yt���I�S�=<<Ol����sh\��������h6l���_�g6�h��m���]�tn�����ao�t,���I��M->�t����]{�����K'G�R�u���������1ed���!���*7�~�fJ���WE�[�$F l6����5���vm;4h�0&�Mx��Q#��R���C��?���mz��#1�E�6�8.z{�|��s����jhhH��i
��XY��5��N����T99��-�C4o/����w����k���/�ii�z��y4�P���(���)���V��4jd��L�I{{'�IP$L����^�;|�W\��[��	!6l�d��w��)�I����o������h���)���g���o����,OOO�p��M������ttt�m��e����O���y{��K%�lX�N]��]t<���Mk
���_*��	�����m��>#3]KK�K��c&�oLjj�v�+���9������M�DAA���s��Wm��tO��z�=�t�M�:����`��
YYY:::�$vG��+���\RTT�h��o���UK�&�q��.++����UVVf2��%R�e�%��������c���]�������CQ����\�XMM���y��	?R!S��RQV��`vFF���v��#G���5�V��cE����/�O����6��j��'O/hhh���P�V
��/�B!!�z�d2#""p+Q�:S�u���_������~�<!D������w
�����4\�P�a���c���?q6( $019A[S���u���2����'��:���h�������g�t��+�r=�{p�w��C�P(�<x�RHRJ����[O�)#'����>������W/�g��j���p7t���>~�_�mM��HE���{��8G���:���<y8�rhbJ�G���n=]�lQ�����a��#�����A|������Qs�Xn�4J�����_�������?�J!���h�6��7p�W��7�&SOW�������4����W/�<���g-m��N��O�e�B
�n��~��?�i*J��m�f�����B�
:�p����D=]=���}�����?��>���P3��[c��������#<�L�5���$4���������"b�S��~���[��~����-[�����j������#	!Wn]]�s��y+�-�^F���z����CW���C�������-�����p�����;s|����7z�a����L����	!���		;����
M���7o���0l!$/?o����]O_����p����'
� u���{���X���&
�zpg���:Z:�[����{����*+*�����Z6m��^�9Y�jA�:�i����h���X6m!���?��f�a���Yz��N=������n�����;J�t#l���#<}��wILI\�ay^~���	!K7�x�����kL�M�>��_�p��%[�o"��)WN9�K��w�]�}��kP��Mu�t���jfv�?�n��p�g��J2�7�1f��������������v��)#'a��G�����/�Fx���p&�pt8�]z�{z���9r�X�~��!�:�'/�=P��M94\}z������edvN�x�����Y�;��Dh��6��[*�e�+���oLi����u��������5����<�bl`z���j`��.^�����k������<"_E�O�>�z~���aC�\
-������RTP����C�F]�L	
;�����o?��U����\4����� ����B��
	!��MJHN4�3��o��#���&
�A��+���d>y�d`�-�[BZ[�Z>{i���Ti{��z��<��������{y0����h�L�c�%�
�'|���}{y�T��D<������kK7����o��U�M�|<� P~D���:j�����_����B��w���y����[�?����sE���z"��#��������=�|>�������0�Z�d���{��7�����p���)#'QI��>hk�z��-�"�++��tw���d2�N��x���pB����r���-]2cQ�[z���d�dO\0%��[-u�>����1�5�Lh�:�L�l��i�����Y��������/@� l~L��d2
�������W���s���{P��6������us����_��e���y��8Q�N�lC.�&$'R�����]�������o����������i/�A];v����X9w�����fG<�{�?����?�o�5����ob�'��'�|+�s�U�.����

;���V\R\P�_�r�Q_|�X�������e�g�����7����{�^�zY���?z`/�^}��������W/�(��^�7�k���{�����.��<w���xB������/e�d�6�������l�t,��%<^vNV_�>������w�����P�I�C�F9z:@[S��KO�%HRR%�6��"�S�V������������CKY:s�����0w7�>+���|��Z���B�h����^�e��~�U��T5VM����<�����;�m�����9���2�7��;>���1CF�V�?�U�w@(n��=0���U[K�LR]U��q3
5
�PH��'M9��=������O������{yB����O�q��M5�����b��,k�����h���9�����Oq'���#u���S��3��'��5o���K�����m�z~���Se7{��������O�L;z&@���k�q�K��:����U��j�B�s�/xL@����\
9�"��/����t������'�P�
<{��mk��o�����n�gph�������o���;mD�~Kf,b2e!|�G�����[�I�e����h�����7g���r�D�>{$������!��M~P���-���[5?�]��G�5�*)(���Yn=]��Z/\���c+)*U��!
�4��4��5j��co�������<tx��1V�,	!�>�l�������������YZf:u�X����6.Y_�+)�rU��7��b�g@��x����+����
3�3y<���?�qE�<�(**�nkW���	�s�/xL@�!�������L���������%��MIK!��twjc������6�
��h��������=�r��uK|<�hkjB�+�)��Y�mM������e*������r����}��\<r����J
��<�E��c#f�:y��g��T�Q��~y<���32�2N�:���.Z�`0Z4i��E�[OWj���O�,,+z�`E����'�&��pT�U$n�P(�1�����(��������_���G3��*�f-�G�#
���>~������(]�����
���9��d�B�����l�B^����;{�<�s��\.�k����U[[6�^�a��M�n6]�����~���Jl�+��uh�^EI�2�U����\�6@QVT21,}G�a�����#�F������Q��������}��%�D��nn���_XP��G[�����=\����"�5��9��Y��t��(�r���?u�G���g�swg�gl�����[?{������8g�����w�~�P��_9\C%kiiUi�:�����/����h�S&/?//?/���i��	
;r9495������s�|�(�r�a�����#n�/,�r�~+���|���k���`8ww:p���;��S�D>�{l_��=����N����F����N��t��D�������~������Gw��:&��I�������=~�$=3}���Y��>�B�7��~2<�?��������da����|��{O������r! ���A�j�+���4��7/��[�[l��%1$�1�~�a���="�m�������y}��?�&�Q������iKf>{�<5#�U���~LM�4f2�}��/^�C������sn�d�&�S�S�E�����v���jfvfbJ��U[�v���.>�KY�aY��M���%���wm����u���\@���~COf�d;
v-Q�Wxr�!��M��3>ut��5���g/������4��}����.	!;Wo������S�T���X���������n��I�j���RQV��tvZF����c7�	��RIm]�
{���?rBMU�w��� �I���7)%i��9����l����O�e�rt8�-WVR�a2EK��<OUe����S���9��&���QC-HS�%%-EW�N�U)$~0�~����A��j�-��Vo�����a�JJm���~XME�rh���f,���������}�������BF�����^:�:����vN|��;r�����4���\���e*����V�[��K!O^<	=pF�d��E�}��8��g�@:���0+K�����5�c<�	���I_����a��K��6�LO�^���������444lll��K
�B����L&3""������������w���r!7/���E��O��_)
�=`������������T�D,T�t��QS�������W��y,BH���4k�������~���U�u��PW�Z�Y R�����b���v3��$z�����z�����S��OT�g��0p���G�vh�?
����~�����MKK�	@Yu���{�/�<y����������C��LC�Z��m9t\CMu��)�F�!$���G_����S��o���K[��X�@p�������

T��[Y4��� �dB��;���!^�����1���������&��o����x�,3'WCM�skk�VV�������7>~I`���0o����������'�^Fe��j��:t��������!5SE�R��*��+��j�U���
��FE<}|�F����w�k���<���{S3����4v�b+'�"�l:�59��d0�UU���:u�QQR�C����>�"��_�s�����o�g���RRw����B�]������B��v�n���d~��K����G��	����9S	!�w��G�O�i����-����'�RVT$��~����9���`u�~z�f���\�u_N����9�zR�^{�y�0o5eB����!�c?
>_�
��\��:�Cn^�<���q�������'��\�$)���T��hEo���]�R����������O-�bo��W��:�n.�1J��'5U?o����fm�oU��(�[�]��b��=��b�u�J��R6�i�*�G�����?D��+���Zn��8��
MIS MMSJ�/�������WoB���ts�ol���z��eYY����8�K���9��j�R�C��#��{x��3/7G}]�����s�F/�.��+w<z5j����vbj���`����#!d����PT������K}���w
���l9��1��^�������������/+)*X5iL�V��%!<# IDATy<��I��=�

O\
�~��SgB���#�>�rs2��D��x�B���R�z&�_�h���Q*SQe�TT���R�����w\��'!V���R���N��G��]������������������Tp ( ��8��	d��8MSH��#P������p��<��<�]�����������N)yP�~����M���$���B����;�o����+���?���f��q���~����J�%�1	����ye��'��(#eE�������F{��gO��`���v�e�������eJ����q8��������YS�~��CC����*6��_��d������`m�����`���6��x>����.��2��X6o&3�KOos4�'��9w��d��(ka����Gw>�p��N���bW�G[����X�W<�n���������zwt_Gi���(v�ggb���G��hz,~�Ji_�TJ���URb�R��3DJDi�������L����UI�!e�*���t������a�C��xB�o�k��0��~��f#��!}����khd�
�G���3�F������������Yu����������`c���r���Y����������%3&+�������a�}�g\�/<���?�7s�i�%�y����K���2��

'�� �X�V>�@�����9W1������$]���^R�u�2�T���R�RPZ��M��MG������!
�"B��������q��{8;2]������015��Y�\}���k&B��L	!�w����6��)#e�*J�/�~����z��wE��5=v()�d��xv�"t:\��X5�����6B�oO��K#����j��
K��J��xv��M�n���l�mz���7��9������k7�����hOqwr�5���\��>{)������`H����}���X��H|��<]/�������Y�p����U_o���S^[8��\H����bW�������K�*+�

��������������v�����|i�XTk����+VY[	!%e�l��,lq����X��������5u��N�����������]��449PQ��J�E/JaU5����������y6��v���/�����I>=��������7�-LMF�����u�T[)Jc�rTI�
J�R�)��:vUI�CJG�����u��|v����g/e����{y����"UI�!e�*���t�����TTV��<�"������!�������PP>bpp �B*�^�}�j��	���%�������Y[��_�u���5���'NO��x�'�w>>#r���d��+.�7H�������L�L��p�n����{�L���|��^�a�}�SK$]��V�&&�r�&^�(e�I�(-(������~��A�������K��/qu�wu�W�f��������BY��������������{�J��

Cz�dV��X7=v()�d�2��;�n��*���54��Vvx!3��������������lM�������L=�����i����������2���Y>E����|n���n��%��D��p��'��_���l�L������3����L+����S���)�9�U,��r.#+�T������e���K$�"�(n�,o�mca>#jG]��~,����0��]B�����{��N��pu���r�"���X1���j����4<^zN�������kh\���3/
J����E���T��,J�#zQUM���Uh"H��v�	�W��&�TeB����y/��!45������a{[����[%�RTu���JU�A/^�"%�:���]/J���L�|~t�I�����#�D��JM�8Z��T%�S����+�p�����Q#�?�s����B�GD��:s�������!�}�����**�l���9�H��o�
S�M�������g��;9x��(���DMb�����R���1�f�mj"K$"��@_����nQqD��KfL.()�}$�I�<jPB�OO���?��������9Wk�&^�4�&U�������SJ�[�'�a����^F�����w��WNLru������� �UJ)����H�u�
����",n��������xy_�A�u�k!��������}�o��PU�5a5?�R�,�_���)���v�M*M��>����g����G����6�����{�uy:^�O��1#w�[����B�>��n���3i��	twr �8����'�g0�W���r��&��t�R&�	W���j;sv�X/!$�Bz����GB������c�D-*zk�!=]�y���
z��~,����yeu�L&���%����[����������~[�Er175
��%���;9�������E���T��,J�QK��&M�����%]�<>b����D*M��34��D]D����	!��}���_RJ��e�*��B���U��
zKQ{��4�:���]/J��L�
K�+�y~1O���{Ujr��z���l�)�L��9|*)9���i�����#7|@h_?��w�E���J���	!	g/\��������E������D��)Db�$�B����h�+*�|�����T}������������Ed�8��2�CY
�JT��g2'�Qa�k��Wo�������
��Hy��&%�]�r��ZJ����.�' l���	��z���+iWr�R�D"���1-r����T� �U�Q�/�~hca^][w:5]qZ�2H���NLnjnn�05+�����	c(RR�v��$^��"%��X�V�������34�0����������H�7o565z{B|z�q8��ky�~��6����P����]����8IsS���^YU�w�^���u�v��==r!����x������o��r�GNVV�X����3S�]����RZ^14D��4���u,!����.�p�Y����X�%IUM����CC�.�^o554�4)s�����ir�lv�>�Ei�Q��]ih�������%���������w��Q�/���J��{���;���b&/b���;�1���Rml(-^�g�J#�3O�X�����wD\���;OK�}�"U���C�*�U���d�{�W"�l?x�^q��y���,���(�|
�F�����<������p8����������dg|����������~�=�N�T������#��z->������j���}{y��B��sy<�NScc�=�!6��5�������z�'��H$�b�>�{�����U%�KUn�����v����' l������NuM���f�X~3�@o���9����U�����e{T=~�?��;m�,j��%R)��>�����b����1�^A[�u,O!!��e���CKs���&�����:\������SCz��3
2�%�'�60655�^�	�����y���;�Z�����%�]��%�BS�P�P��Wv>����\OWw������T�.�L���#*+4x&n�PU���6NF�������2:���Ic�%�|���j�XTU���Ly::��K�����r�~I	���T��TE�N����#jk�iK��!j���4,��t����	�=�.df�{y������S`��6����H$K��anf*��V��V�G�n��*E&��;s
�*��Ao)�3DU��'tm�EetD&�����T�C���U��S���~%R��}k��_]0��@����(UV��	�|�{�����
��(�������k������HZ��g^��tu����Aw��J���b�lK������i��In������w�}{1ko����`����������_���W���0�.]���(j
�����##$�Z�oOw���R����(��!�RP��
�r����8����e^���������f�XZB�|=K��X2����������>���������~q���D���R�4��[u��U��NP{[oU)�u�PU�]�)#��i�p��/T��|�����$���1���cFz�>z$���;���5������C������������%���l�����1��%�8��BEM��{E%�7�����t�tu	!�A��*��[aI���=������^�VB3���^�}�A���
��=5~LM]�����^��O�WG����9+sa��<'[[}����s�Y�X?�^_,:"v}����D/��`���nz����7�;XR^q5�����>jkR������#���KJ�_�,�[E���UR��R�����i't�zQ!j;��^U�=�P�JUUj�C����o|��;E�.���pE��0gD�b�T*#�H�2�T*jj&�����~�B�#��m�A>!
O]�|D���������v�e����{{z0g�A>^g.^���docUZ^�����K11���}��qkg��KK����r�����264$�����}�A;OW���n�fe/�:�y����3�$$���).+OLM�?�YUUS��hld`ca~$1���>,4��!��x�����Ji]P������jS��a�������{����]���?����3��$�� �U-��/�~����������s7)�����j)�MO����Si
)Y�'^m��E
�������R][WQUm����%�>_o@�������ZJ��w�]'02z��E��g���t�}�y���5G��J���?D��7&l����[�c�/�K��R�R(�xX�����"�X��<��=��d)4���"���o?x���{o�����Drwr�2j!����tjz�������s��{��EI����bW�a����'��\z�8��W��=����>�-��GF�;~����������8��2^���\D���|�\��a�%���2�Jeb�����C	Hz}���X�Q�JC~��S�p_h"�p8��eWZ�E5lz��^=�F�������|�.�k*0.*{@���I8�����{��S���=����X�JU�B�y�Q������3DzDu�	�^T���M!���R���U��R�(���~��2�>�e�|������!�n�����������I>������PP>bl�C}���S�u�&�F~=���m��0>������Z��Q������q�W]c��;_����	�������p9�%��3"G������	K����Q�<�
���5-&�TR�eC}�����~�����WY]�5�P����������W8Q�u�2T[)���RPZ��M���������3�IL�a�.#�o����(�BY�}���g���3��G��j��&��#������������LI<�CvN��K:����7�2b`����c��������K����������AjVNo����U\.7������a�C4lz\g����7n�{,a��H����-}j��3)�����7��{�p�5�y�d`/������s;324��5��MV������f����pz���s�b��]����6�D���*�k���{����N��]�ug�������UyR���������;�z�98�h�������(1���N>r:���S�{:;i���tD��(v�������~��F��eG��(��1�;��7.���aa�}��1#������_����G�03��<C��W�������}�OB^_<����]�TU)���URb�R��3DJDu�	�^T��H+g]�*��q(]E�Jv�0��G��Z]\]okb�b��V����	QA>^�A	g/$^H{���q�������5o/��O�}�"2.����Eyyy��\(8P*�Bd2���)B�.�����E}��;�u�y_P��tc[cm�9DY��e^�L�J����������.�b
j@���
���	S���~�=���Z��[�L��^��e�?~h���5��k�^�h�w���}�g�-5��w�0>x��PT�E,������i�#��LZ�����g��x���Z`l�����!���Um�]�X���;��&�L��zmS���
He��_|F�����J�����]����316���=rP��+�S.e&]�TQU-458�O@{�!#����3�TV����yG
���#�*
U��z�����lk�[�.R-�
J�[��
@� l:MR��}�O�^>~���~}	!�s����/��464��3t��.��f����%�.�cf"�rw�2P`d���[iy�������t{{yL��x,��j!�BZLB�����5�)�H�z�fM�0��(YV�/��!d����^����c���S�S��q������W!���=�tVq������11_������BN���������,��������>Xq9��9��L��������o�}u�S�1!��7_%�\������&@��|81�J����Z}>�����aa=��4l(��y���x;s�e1.����_�B���%N��K:��kK�KfD�t��������Y7s��L����Bk��8�jcGDa���x<���/B>�i���Wm��n���J����x)�z���J�'�_���]YU-455��o/���*;�#�OV��r���x�75+':���q���
�K�8���c��(���
QSS���C���jk��&J�s����F�>_O��(��'�������IvV��������pF@��z=&�����.vw�w<bdh���:	)�O�^�=n���eqY��G8����Q��dP*��EOI�2�;7��v��S�Y�����$6a���4!�}|z�BJ������i���-	!�������7�D��4k�������O>��Y0y�m�_�����iIyELB���b^�?�I(��F�O;vx{N=���~��c	g/DH_�Z}c��������}���Q;VRF"vCv;�N!�(Y���]�!��p���rV���fM��=�_
������}/��N�^bf���Bf���]ZvnT� ��/��M��`8{!���e�f2�����6�-_|�9w��d��(ka����Gw>�p��NJ-��YiR��9���U��0�xX�baEU���?&>���������5���r�`��G��hz,~�J	QM*�u[�URb�R��s=JD�%�OJ�4{\���U��[;c��y�8k8���U��L&S��=�U����@7��-���?���7!�L ����v�s"DY���P|��f#��!}����khl�����o����9�qE��(��{���[[Bl�<�\�3�N������j�y	n��:���w4mM�>�?#jd�nLix����uP
R�A���o�ffWVU/�1Y�g�-(-n�66a���L|==��!��QD1162W��0.�|g��##!���C�����f1s������@@173%���������	��?��elh����l�3�����j�����������
J)�H�n�n�)�%��|��7�t�\��j��7��+m����)�2F����J��������5����{��.[m�cd]�q�����Md�X�y��� 7G{B��������e�����K�U�FC�����L>����G�3����x����uu����S�W<��z�������9V�BJ�Pj�M�J��%���g�\�|PYilh���g����������].��3�������,�kma^^U�ba��*k!!�����M[��-�b�k��74���������a��Qz����b���5s��L?�P��J�EGDaU5����������y6��v���/�����I>=��������7�-LMF������m����m�rTI�
J�R��(��supp3_; �?�~A�����~�
���JzR:���5��[���/�\={)������Cm�Ve���C7��mqIJEe�����Um���/>���C��w��Z�X�����#Z�B��eU��������%v�V�e��o�e�,#�^q���A�w�9����)����@�H���@��T*�~����[��wP
R��I�����8=-r��{��-B��k IDAT(-n�66a���:����K��/qu�wu�W�f�b��q����w{8;��<�u��>_/��wXh&��U��76�54l�]PRflh��gpp����)�H�n�fMU
)�@���|��7:��������.g

���^��vwv��0���~1+��.[��w��t��#SG�vwmgl,-2r�z{2���pq�O���g�%�[4u���}AI��=�::\&����W�o/�7����������{y3�������u�J��^�mjVj��r.#+�T������e���K$�"�(n�,o�mca>#jG]����h����;w	!w�7���;!������:���boc������[[���x�9�SF>���q��?�\�4|@(=_��2�f���S��eQ��jz���BAjV��M����"45af�R.d\�{i������������o��:z���j�U��
z��:��G�D"�U������bV�&�o�������D���G���p8K��)02K$���U���=����od#����%%3�F�iU{���v�r��Yw'O7�����(����UT�����sX��P_B�H�$�

.�\�sh����&�D"����%����Kz���F������ZS�z�$����kpQ�����V��6�]M]��c#��,)�8����h/4<	%SYU}��8�_�%3&���>�$n5�?}Uk����!<4x\���;�b����W���4��4�+��y���<�_��T�BJ!���4_���� ����9�"���]{�J���F�"��x�{t��z�<�M���v��h]�N��G��1c�����V����������������L��A}��!��6a!�����E����~�l�	!C�����l�Uq95=l����f��x	!��C�}�<zB\�L[� jQ�[�������L������E���0�����d���d2����������p�<��o�9,!���!���#}w'���R��b��Y4��v���D/��h���HU��p8�~I/�����#�J���
y4Q����>#CBH?��G������~YG/�R�����JU�A�T���#���{Jz��GOk��������

������;"c����w�_�S� Te����a<������������//�6Q~�=�������WTr�����i+�	g/\��������E������D���E����6F�D&{��W��1|@h_?��w�E���J����Z��%��i�GD����J����.�' l�����z���+iWr�R�D"���1-r������l���-�f%*R�3��P)�ZS|���yum���tf[��e$j��}81�������������&�a7�S
��e���n�Q�����k����-�;C�
�|��J��{����{�VcSS��'!����������Km����������u8����475y����U�yw����_�mw����#GB�**�'�;�|N�f.�y�deU����|93����@����f��xe���W
Q�(����K��y���K:��Gmj���B�DRUS{-��������EM
�"M��\���ty�<=�][f����2%z����;"J����t.�F���Gv���FQ�?f+QSs���7��k�9��c$���Nm[VzT�66���s=�6���~��?x:\;k�/����PQ��N�E	�C����oOw�����jUj��
xBt��_�D����{���������pU����$I���9B����l�:�6rP?;B���MXh��Sg"����ty����`�^^n.����\����kO2d��F�����<�������{��+%����f��gMvE)(�o]�O@�@kc#���:���b��v����Cr8#�'�M���w�%��X����273��R�[[+���z�TJ�R����Cvxhp�)�����{���Q�A�e���J��A������i�������;$$��������������u�?���S����R����]g�}-���]���.k�d��I��o�����+��cma���;f�@�[w*��L&S	���GQ����
z-�kV���0���h����lgej����M�a��q�5s3S��N������%3&��\�_R��p,&2UQZX�|�n�mm���u�JQ����ai.���M�L ���v!3����|�?���G���W�D"Y��s3S�L�b��j?�u����L��LJ��Jyl�{U�z�Q��7�8�D�,����'���^�����4��L��Q�j�R��T���O�J���������`��������F�HTY]c&�����naqQi���C���bI�X��w?-�6�M��#�b�ZO�=r�g{�������U7��wu��p8w�X$CF�'?�688P��f�W&�v��ed^��tu���5�/JAi}+��}����jo�y=O~�_�6}�.�mciA���,�fO`�8��^�~C�x������������jm��8_�������K��&:\�����)WG{�H����a��m��U��N����>�S
AU�)#�&�:\����6 �?.������I@]Cc��[3���p}�H�[�v<ZUSk*0�7=��	�ow�aB$$�;9�[[�KFai�����'�c�Kq��!�4��!6���J�o���������BL���U�#���R{B}���QU�mjV����L����o�}P�p��%�O�SSW�f����3�
�a����n���2f^�s��5�����x9���������|�h���9�Z��D/���zGDoz����7�;XR^q5�����>jkR������#���KJ�_�,�[�I)=���
�xU���#�j��FQS`/O!��|{��MF���B�vD�J�U�� �z�7>�����W��r8
�"B�C�s6��6���{�6� ��G�4��R��"���R�������s��y��%�����r�����264�l��p(��|��\����������"9=#���������?��`���|������ES'BX'#������V�Bg;��������==:��_J2�V����`_o��UZ_�
�#���=�a��D�
�y��=G����kh�w�����|z�
<s1�@BbXh�������a�C��Rj��x]�hK��;w��.��J�8�PF�6���8���u���,��������FQ�HT][K������6��[�E��7 ��L�e�K-�R�������"M��suy:��������	���N�e���a�6����-������i�b)V<��}��qC,�LV�JL����XB�B��<�����W����[��;9L5����q:5=��������T�m�)����
������oXhpt�I/7��.���;c��m��lk������pu�����?b��,�ke.��w���������Ty�K$��Ue2"���b1!DGG���|�h���9�Z�����)\�/4p8J��+
�vD��G���]`d���	K�Y���`�r��������������|����=ej\�#=
�u���z[�U�
J��:��GTUMmLB���������������`M���zQM:�6��X�Z9��N3z�������6��O�&�e�54|��&�=]�U+��W�i?�n���+��������I>=��_Hd��s���V�U����D����
���=G
�����}F��S��?a)4�5���;�-�d�
b���������������:�)�P[)U�5��m�EUAu�V�|��C� l��N����r$1��m����=���XL�����p*)����~xhpx��jW�6yT����#qU�uB�������A�2��Y���R��L��~��������2�7��&���/*��X?$$�tj����E%������j��������=���\n`/��Y9���h�����IQ_o���X��q�,����[���#gR��[[_o�����63j����^����q��v:fdh��k��;s���aU����	��puig�Pj�M�J��
��m���OUV��	���1�_�����v������-����c7H�g\V���<�Zs'UO7�'m_����_E"�����GN'B^|jzOg'��u�������e����������`d`@�^v������C�}���O����3r_�������lz�3A��3�����o|���m����w�!����l��E��J��e�Q%%6(�K9��DT��������DMN�6/��.��%���j�i�,��Z��v��1��G��Z]\]okb�b��V����	QA>^�A	g/$^H{���q�������5o/��O�}�"2.����E��q���i�B�p���R��"��_0��a^p����$.��n��788�+���*����shk�!B���/�:fjV*�������gv�kP�����l����M<B��QT��c����O��i�]'1������/@w�?<�7"��'ju��?�7"��B����5�5�:�vlBm>iDM"y��u`
�+�u������}��������F�}q�:\.!D&�m�s�_����������/���m��Z�"������������6N\4U*���<��V�d�:�g��?��m��L�=k�f��ESn����7'��-1��D�y���D��������q#�j�eU�����%�t���#�R��A�t������U���x~���!GO�m����n����F���K�|}B���fg_�!��p�����B���E��Rn������U�Y���#�s��|��U�c[���j?[�l����7m�+��Y�v|g�����BJ���g}(BY�����	!:::�H��m_��j�SO������)1�������������si���y���5!$3>�r2��+,�0a�
�k7�p"�diy���x����/�.0t�����KM�(����F���P�!�#L��"��������;�$_����{�uh'Ci_�;�V���_{:��h7g�N���v_��M�M���O?v�2r�m�J���:�)�M��U�D"�y���=�x`mi=v����=����E9��dt�w��#:XT%@W���~c��r�W���I�����7?{G_O�SOB~��a�_;>�SOw��o����\.���^k�~(�������[�C��2{;�����������F�lEI�����?}��[��
���z��w�����FBv���T*��d��/������e�o;������y��L<w��/?�����a�['^���%��2��V�� l6�f��	a�!7��|����}����!������t���W��2�o~\TR�����K������}�G�}m�+�bq����?}���Ww���$ZMm����6����-�,��7�������(Z�)��T�T��������o���c^P�wvM�J
)Y�$������m1���>z �7�`�����}<�I�y�
;6��I���ff��<�W�p8��G�|z������rur��������[���O�t���n�PKi�v�����Pwg��E-�9�*.�6vJGw2����s`�ku&�}h���G�-�V�I���yJ`Sb�R�?o�u��������k���y���"��}q������di2:��U��T*U���*5�J���O���wg���Q��Bl�lF��v�����X�������2���k,�C1v�3�y���e�?��l�����O�M\0m�_��i�%[�n�:v���Q���V63�����9�U�Iu��������X��y(�������!��?-=+}��?�SY�YV�x�cW�,SP�P�[��
@� l:������!����������|��[���}{����^B�?�lk52A��X�B�!/��RAq����U{�37�`����)z�_��3�U�(S(�����Z��e|g�������eJ����]��~=������?7+5�"s�6��gN����u�����=i��qq���A~���`�>���(��
��rO�_���&fs��Y4s3�,j}��'q����I�T<hhl\����E��(�r�/�8:SzQv��%���������w-���G�{q���'��k��.[�������
�)��D��CU���
�[N�2��^�qm��D������W
�MLML�SS�+����|������b���X�Z�!(�^v�����m�����g^x���������i���?a�����1�������?}s"����*;��_�E/��Rm�(�yJ`Sb�R���Y��{�{B�{z
����6�('YjGmU%=)Qqi������������-�,9t�w��lUt�x���g�������������f1/$���sI��x�u����yT��KI�Lo�V�$zo�G.����([�J�L&������9�U�~����H��7$������5����DZ,+�T���G;�I�U%^�X����YVE�2��Vjc6�����!'/w����6�70�7P���.!���<	%s6�\H@��_��OJ06D
��?m.sUe�R���~�j�*�.}���7�Y��O=a����*U�;���%��,SN����?��?���}h��1�2�ek�9y�o}��{��3(d`;������S�F��d�����Hw���u���?�6�/0�����,��x���%�l��=�B���wwq��������B��m����!��u��^�����K�{h�w��~��������w���77��?�����"���,ssv�x����/U���P7g���.B��^Y��������~��c"FB�zx2�������������[�-y�a�����n���3s����`�9�����D/�����KU�2���.�h�[/�����C��m���"����1�t�k7����?��go���M�	�VI��'#�[Ul��72b��
?d_��r��q����s/?����r�E:��tD���#�/~\��r�X�������	U�	U	���#g/��o���]����_��*pd���\���GO`����_U�m�9���������d�5���Df&��O���>cfb���TWW������-�-G!_2���]���w����{(������dYU���Ei�����e���4� l6����r�Lfea����w��n�!�7�����wQi���Cc�/��=3{�/[�m����4 R��,?���9q����DM^�����L�Y7���O���w�o���>��=8��P�B"���7r,!d��1q��E���|iy�K���������T���?035�`��������Y���v��s����}�\�����i�v��fV;u|����<��z��S��/����q��o��������s���x	![�l�9q��S�i���7~���3��d��o�|�����=_�"����*n��E%ER����$�Lv�|!����U������������q�s�9�bfb��'7/Wm���,:��Z��eQ���4iz\.wJ����C��fB�X,>;m�TfBk����6�vut����5I,3��`�*����d�u`��
z�N3i���3��02x���G��<f�����d�
���E����e��[?��;7'W����%����3��{d2���?��������������G�K�|��u_I$�����O[e_�>�x\�����3S_O��P&��>l�����=[�^����W��\��b��)�<]��#���}p���jO�;�r�fY-Jj}+@� l�>��>3�������I$�fqs���^{��iG�{�1��uvp�y'��������M"e?�}���m�.�ee[���x^����������[,�9~����_��9eCJ
����k�P���3u�u�]1����p�N'�9Vm��JI�<f���S��Q�'�`k���K����?u��o"4S IDAT�WO;����Bn������~���cy<�����"���>k���6�L&�u���iO����wP�m������N'�:������j���*nN��������)s��9z�xm}]Mm�����w}���5yl��]���^K��eWZ?r�4��Q���k���#�?�r���fj�df����o~����TQS#��%����:�Q�J[ml(-�
;69yt����<����V|���.o�������d�MF'�������D���kQ�]���_[+�?���t��i/���o7���;��s������g��P�������_���/k�����M����X,f�jelh����������F�e~1���7R����I����E(�������j77�D#�5�~���(�����wf����*������
���4�����>(S\�(���7:|��E/r�\3S���T��V���B����*0�T��j����~z���xX!��k�����������L�V�T�g]�����&�)$���Q���E!�Gc���G������p����#�������>]�Qrj��o-�3z��>��aoc7y���c&�;����O����f�o����sn��PJDqnZ"���xD�SE;h�hS�BtuuUO�BH}c������V��jf	%_���"����*�z�z7n��z���_��~�����:\�����m�NR���u������^KC��eQ�Nt�����X���H��!��:�����OB���^mnn�����v��x���`�*UU�L*U{2�I`�c��R~���s��
��'��z���>������UE�$K����^T&����,[��U�9U	�9������e�.�_�`yye�������)���1j�����Y�(�<�D"i�~�j�j�K�m�l�����z%�V��O�b�5�D"���a?~�6�/�Ev8No��3����,I�|1��_q�xb|��~#c�&�����F������d����#��?�u�;�r�YVU_������.�' l@���&a�����;��Nn��?m���N164rvp~K���wB�I�L��a��w�����&��j��H���_10<�����w������Q�wM�^k��K��_�����V��KW.��r�o��Dm��&�&�=�������,�t�����W��������6vgp��g�,^��7}zy��b��k7�}���o>\m!�`��y�Bj��!n�n����o.�,7404�7 ��X��+��,�H�Wo\��'�E�AG[��������7�M;w����	?��w>PY>��Y{b�N7��/z#bw|�
��uqt�;���k"0�|��QG{G��#����]���^KC��eQ��/�M�2}��W>X�'?�B���y��$�������\s�����2�Q�RN�����M	J�J�R�T��!p�\�����,J2:��p8j;"Te'W%@'�����m��������R��SS[SS[SSWK�p8c�En��{������s����~�����@Q��3��&-���tBqa�HT�P_�P/�H$R���R���o��c�6�f�-�����j����^>����X8c~����G��K���}������������"_�e%}�q���*|��'�+�O�K�2=�Z��4(Y����2��V�|�a��,{z��k�}�i�������~�_P(3�$�=iVaq�W?SRV�t!y��m�&�P����B��'��/�������n��c��j�w�MO���4|���R�LO����M�����%X���HS���C�����/������o446�K�������W?|������eYW�|��kgg����3�9q����b����[��x~�ok�
G
q�xlfnVm}���?�@%�m������?mn�����4�s�r�������{psr}���V��:��
z������!������S��'����/�t��n�M�M�&��I$���1�ZJ�����`�9h�k�V�����P���2%z���V�/
�!dh���B��������_P�<�6�����!E�Evl264�����qQ��"y2��u�TU)����
lUaC)��a��������H�7n���+jX�����d����zQM:�6�(���U	����W���UY�0��q�%���#����S�U?~��������A��~��~(f>?�����?C�B~����Cm�l�����u�\��V�d�����������W9�;}����C���TY�u�_dO��\TR��'o����z�lZ�^�u������R�u�J��e
�e���(�B���A�t5}������7�8��Ef&���F���e�������r�W;��451]8s�����j�>�P�����L?}��'���l��zi%sq$}|g���Q�Bz�)�gw(�J��������Kttt"�G8v��Y�4�y.�������O�������"�|�������u�?ZQ����X0���OV~�<`22|Tcc��[{����fc��Y��Ef�E�����K�=i���~��3p��x'��T[_����E%E6V6��M}v���;�2fRJj���W���O}�����G��C
wg�����BB�������;�1��N��O���i���B~�vcH`_�w2��]��y���������I��BS3J����������=y��_������oO���?�~U���.��.�����J�3yv�����}�����_E�k�n��^,O�TT�&'#J���]��
��`�G+���,�-G��zq�sj#�r�EIFg��vDZ9�FU���u��p�=�?U���Z����M� !������}2��2���7!�'�t5�b���A;���
�dnH&��_�d2B����&%%qQ�E�}�������II?vA�F�������0��]e&\�����<+���q8��/��1��������0��%�P^f���������8P*�Bd23�+���R)��YEIJJ�U���|�WqX&�q�\B��a����
���������M�����?v�X�{p�/@�6q��	&(.?~�����`���S��m=�Kp�g�n�����d���cn��S����	d2�T*m�
7|��0���a�����~X�������p8�_��������n�SYY�R����_�O�x����^[_tw���S������6�@�@w��_�n�w��y�@��#��=���>D!h]ZZ��a��R)!D&��d2��T*e^3�!III<�@�'��U���d\.��}2E(~��w]y��Qcc�����7#�O���$�PXUU����hm�w���Be~���.���k��9�+�P����2��������w�mhhh����]\\��<}}������"kkk�����`B�������{\�����N��P):#��R��S6��3a#�%gkj����P���$�6}&���c
9���������|~��/|Jbe������\���\����w{x��������S��U�C����7e����[7l����100���Q���/_^������EAA�k����uk��Z��M��?����PS�f��7�x������}���%%%�C333'''__��~����}S�!��K�`ee������Y����k�v������}��k���A_=�C�	o�FCcjj:{����uaaa��\�������U����t��?��.2���!���,�;d���'O�<yr��-B�M�6I����x�f������[�lY�|����G���S��{���??))��?|�6O�<�R�*re�K�T*��g���A���9s�������6�9�Q�q=��
�i������w������9r$  �9�Q%����Q��������m{]�G}T��Y�f�y������/_>��������o��i��yR��M��N�:y�������7/&&f����������_���B���k��}/����%3)�����������?��q��������rgYf*����N�:���>�����N�:}������/�����Y&�U
IBB�����e����unn��}��u�v���2E2C�y�j4�q��C�4;v���c���R�III�G�:u5j��Q#KKK!������n��B���lSS��/��7������t���R����I�&�h������7���}���Q�F��5��7�l���������5�_}�������u�-f��%%0~���6m�|��g��7����;wn�����+U����Y>�����m�������]�vK�.�������f����:u���tss�������f�o%���w���|��]�&�����*�������q����u�3gNjj����uCf\��_�fM������������[���!?��c���o���u��a����:u����t����L7�#/����oMMM*�����A�FZ����/���_�������OGGG������������o����!��������������{��w�������u�:88����6l����>}z���������|���
4������o����c�BCC��e�����>|�������{��7|��+V����t;i�x��-�.]*�;AQXX8a�GG�������>�yb(3d��Vy4�g��9s���e��������������w�}��'N����h����3����g�2E2C�y�^��=����h���%�?k��v���x��LM)�����������W�^hh�B���kW�
=z$EEE���c���vvv			���o������B__���_|���3����u��a���AAAB��.Z�h��=...���>>>...>>>zzz��]suu�������/�������k�����7����ez����dLHH�7o����===SSS����O?�T~\O�����I��7��k��]����_|��M���'_�|YjG__�����W��������t7d�%S�����5kVLLL�._��������x��/a���5k�}�vooo'''!D�.]�t�R���t�r���L7d"_9�����9d�����
2d��1R����'N���:��v�1c�H*����������|����WfffZ[[W��
2���';;���F�w�JU
�KfR>��S��3g����^�z����:/3�O�J�Z}���q��i���������U���zk��Y�p��
�x��S�NIh�MV�X��?������y��u�W�~<I�Ld�����EDC�q���;:;;���c����hT�����������y��%
���}8u�T����?����y�!�<lUN����zy2�z�C������Y��s�N�Z��o_��������gZ�+������7ntrr211��500������������
+..>u��T�P(7n<t�P!D���;v��)�y���������������RNE�R��u���i���"OO�&M�(
)A�P(��y��h�e�d\�|���o��]uuu��o?b������;�����Q������U�^�����:���G��������x�bxxx�v�lmm+�
�q��Z�|���C{��mjj��m��k����/a�eDGG���Iy�O?�������H���������
�'J��������}���`�T*������C�~������f���d�Middd���v����S�N�������9s��!C� **����S�L�����'����n��:_2�����e�cc�-[�|���.\�hQEf��S���[TTT�~}������=|��E|�V�\9z��:���������k�,���4h����[+�����7j��L�����������h�T�m��
4H1p��;vhv�(�����d���FFF�V�*�{~�hHy���Y�a�r2�F���52�z�C����Jv��Yf��n��T���Z�j�����Ue���3g�<z�haa��H���HS��qc�������)���g������:tx�����}M�PQ�~����KQ�n]�B����Y��42
���i._�<a��GWW�/��R�R������i��;g��{�����*g������%%%�=�������_P7�V�V�Z�.]�.j�����4j�h���YYYG���w�������5yb4�j�L�+y�!?mR�U�^�z��U��u��i����o���b����`!DzzzRR���G�V7((h���)))S�NU�T��c������U�V%$$888T��I�J	���n������Z���!�]��M/\��f��I�&��r�Sibb���nff&�6>�S������,X�`���zzz���:::��_����V?Oe�\�3_����o�������u�����s��A�*�k3i����������'n+
O���V�d��r��>k4d��2����_���{Oz��Z�.--��uT�E�/u?�'.Z6l���gw��u����g��)}�?^���������7���=<<v��]n-2
�����j��}����,�z�6l��v���jB����q/��*�^�~R�=���?_�zu���/��������YAC�ggg7l�����:::z����.]��=����|E����/T�f����S\\\\\�x�b�U�O���
���}��G�f�
���=z�h���III{����a[U�^�*���~V�w�F����XYYi�����|������Y���:u�(��;w�����������w�����<<�W���022�/-999RnO�Vk�*���r�|<>d����h$$$���6n����%??��������k�����S_��&?��?l�O����}�/��2�j������w�����T*Ujj��#��1�XRR�������������r�����L����B�ptt���1##�������P(��m��d����NNN.\x�{=k�|\�z��5m��I�&�U���'�G��B�ppp�~}��C���]����D�O���������y����<����:_����'��������~���'''kw���NOO/--����c��uppppp������vpp���_�ju�-�w�����@����~������7o���gp������Z�
������S��KfR�m���fG����������O�e��T(m�����5E)))������M��z����������iy�������+����	{��Y&�U������}tt������>|���������]�jUrrr||�s��<�2C�y�����Q�g����^��Ts������
!tuu��������_����[�nEv$����s��"''���������****,,,,,T��*�J:���4(S$�����~��G�Z���?�[���?|�XM�6����������������������a���������s�����5��d�U�7!�&�[��<���L-�M�6}���'O���������s�*�����M�6��e888\�zu�������[��?>m���M��j�J��L�_Dx+�D����c�����5$$����7o�LLL���
B���k�O����---�=����P(�t���;wfgg>|x��E^^^RNh�������+H�����}����������W�^=z���Q�~��������FU��W��KfR�c���������o��q�������5J~���r���			7n�������7m��w;]� IDAT�GUz*��z�����o��}{qq��K�0s�L��{������������y��=��{��Y>�U�m�����������SS�-[�T�����i���������{�_���������<�JURR"K�p�!�?l���!n��qM�T]��(��FCf\�C��"������,]�t���...QQQ666����;V�Vhhhhhh���>|hbb���5}��r���������3g��9B��;wv��Y�A�"�����t��Mkk����a��=O�:v��u��9s�����~�������*}/OO�a���=������r�����&|"�{YYYi.�<y�����G��HV�Y��i���5l������^�~���z���|��s9===//o���\~g``�k���s������{�����w�]�ti�oi����o�>Q2���obbt��-[[[)2B���kk�nZ�^�Z�ji�������N�:5''����w����M��V�^���zxxh�jW�:_�m��m���N�:m����c������QU��0_��bee�}��i��������L�8Q������T���3**�����<y������K{����{�~����_PP�p���'���
8044T�p�����+�1c����[�����!���j��i�&///�R���CAW���{����qm|}}<���w��!�]����kW�����7o���G�!�<l���K�E�����733�y6���������&�����899Y������{�1�DQQQ��/����N�����:��G�B�^��h�l@U9|�����f�Y����B]�� )))c��%��|��$D*�N�:���>Qii������gI�/�J����X�r%�`��T
��C��z�������
��z:~����}ER�g���!^P������/�x�~�����o��]MMMMMM������BBBrrr����������@���M�2�u��
6ttt����t������������G\\��|JJ�����K�x����x���A���9s�������6�9<y��J�z�FJKK��&((h���qqq���[�~��#G�����O>�d��!���>|�g�}�v�Z�G�~������7n��[�9s����^�vM*������������k�t�R�t�W_}���amm��E�Y�fir�!!!�{����7����G�}����5rpp�������
$egg���^�xq��y...���#G���+&&��7����j�����K�j�t���c�~��7���:t��t��N�:IE�/���BBB\]]�����h�"ME����w����h�7!!a��yQQQYYYK�.]�h������<�h��M�6��qc��}�����m�T��_,[���������������;vl�����3����|�����TdccS�^�������������+,����Y�p���������7._�|��RQ�f��o�����K�.���B��}��������f\���Nvv�&	
�8�'�MJJJJJ*s���������^�x1<<�]�v���B�������v��U��}�#F�_���O?B��ySWW���Taccs�����9]])��T*�a�����i#��<y�����K-,,.\�p��q}}}��+W�=zt��������k���������}�Q�v�����t������o�^q��
!�����kkk!��[�^�K��\O^��x���y�������XZZ�o����������_�|�U�V��\]]�]�&m���OWWWoo�������W��7n��d^uttZ�l�]Z�V��={���J�X�`�f��3f\�zU�z�Q�F;w�<u���������"5%�())�4"m��P(xzTO��Y;�[������;z���?�|����7ZXX!�j���R}}})�jll�g�������7�LNN�����{w�7*--�]��������444����KKNN��%����n��a_�utt���k/]�$
�����k�c�<Tr����R�������i�&M���[WsR�P8::�:uJs&##���A����������'''����\�R��odaa�y�nii��3g*�=gg���t�������|!�������rrr4E�������
4���:v������~j�����5T:������1~������Q�V����������+�����s��"''���������
6�����o��;w.^�x��U�|�������K�./\�P�����������o/..�t���f��)�ppp�z���#~���[�n?~|��iM�6m�������a��n�*��/]��i��#FHK�%7n���E�A^��s��C������y���:,,l��aRQhhhhhh���>|hbb���5}�tMEOO�a���=������r����+~�\���{w�R9f��w�y'//��n������`���'N4338p`hh����`��]s������{���������t�R�m��|���s��v�jii��������'k7[&;��������lxi���=z��t�{���S]�~��|`oo��_���eff�����[�^�f
�
��������2��������={V��2==���j���*����#G������c�&M���w�����x�����B5�DMM����k�g��6m���mHHHDD��o���o:r�������0�C����&j����
�Wl���}��&&&�Y�����o
�������v@�PDFFV���j��]�l��I.\X}"��m[�W:::666>>>������$..�w���>//O~'�gm�<����#Fxyy	!��=��O�;v���
!���_�����;o�������/���������
��{&JKKut���O��1c�����4�J%s��6@^�X�[�n�&M�4i����Zaee%}�W�^rrr�Z��^�������zzz;w�B��������5������h���EEE�+���[�hahhhgg�`��Z�}���7+�������������Q�F]�v�?��S��\�"-^��i�������7_�`Aii��k���&M*�HXXX�N��o���#��
���I�Ud\�/vtt��9e���kLL���������}XX��(++K�P�;wn�����
�b��AR�������Y&�&L���K^^���fy�L�O��b������

�����[������U�w�������e���3k���������B__?99�����?�8|����?g���������������������%K�Ta����Rz2>>~�����<����3g���s�-[��p�B��g����4�������o8�������k�.M������7g��;v������O���			R��U�BCCg������i����WO�>]*���366

���_>����o��eR���[AAAAA���s��D~��ek��122*�[����m�i1B(��������S�N>|��/���h���_�B�z�jiQ�Z�^�j��~(�H�P(�����������m����m�6���e����===utt��i��r�^�222���+���W���=�m�����B����Q�Fu��MWW�s��c��]�z��J�<x���}���B�3g��j�J����D___WWW}}��'jR���������533B����8qb���RQTT���_�~������}���D���������KJJrqq155����_�T*���7����������B�7Mu��C��������������3==��.�M��wq���%%%[�nB8p�����G���6o�\�Gtpp�����/^���^x���W�\��9����U�V&&&��������chh���dnn�c����K����4����gff�T��-[�������6m233��e��G�~���rW�feeIIe�����Xf\�������5z��w###�]�&UQ�����o�������;���W�QWW�o������4/O�|�<-��G{{{M���!>e���������c�/_.�X�r���C�����+�j�&idd�d���G�Ik�+��>8q�DDD�����E������TC�T��U�e��*������v������g��T*U��~�juII���$�n���������?���;���k�.GG���$�V�������i,��� <gu)����4����^N�8155���III'N�.���?��5j$7k�,55USt���2KE������{��U����stt		������Oqq�B�P8;;koA|��	'''�Bafffaa�y�������6m�411���J��Y�����+W�c�Z}��	M���JJJ>|������2j�(���
�������3�Z����~}r�<-���3�21�+
TD�H����8p������sww�.�}�vTTTqq���g��[7h� �|HHHbb��-[�����?��{�)S�hW��g����f]�3Y�z��+W�M�&}���?r��Z�NII���

��Z�l�~�z���}����aC�K~������aCjj�������sss5E2��4iR��}srr��n�:{��&V��������{w~~~ZZ���#�_�^n7���
�j�J���5�f?-�BKK�������.''���s�4�L�21P.�������������2��y�����[YY����={���A
ajj��g�^�z-Z���:cii�r������W���7u��q���7n���#G����^��I�^�z�"����'{yy���������G�$\e�5k�,���[�m����-""B*9r�������LMM?����c���5��nXXX\�x���?���9"y!D��]G�9d�[[�^�z�����6(C�R$''�����wt��-!!!��������~~~w����k��y�E;p���������5������g�������^v��'����i�}��I���[zz����������j@�w���O+���g
�D�Vt�t�~���%P
)���7o��j���� �[������ww��E(��:v�XZZ*�P��R�W�V���J�R�"%%��l�|��!�Bq����s���<��p#�Z������oee�����[e��D������MKKS�TO,R(Ux��s�.]�4>>������3���:::_|��|<������-Z���-X�@�c���������acc������'=z�h�����������O����T4a��.]����)���������^dd���������o~~�vO6o��P(���+�s��+W�������G�=z�8q�D�E�Lj@�7&&&"""666777)))**j��%R�R�LHHpqqIKK;u��������K�h��e�|���}��_�����i�&===M��5k���
���qc���;����:uj��};v��4X9}�Q�~��*�*99999y��a��3���e�����{zz����i�&888&&F*R(���~~~B333OO���t�h����
j���R��[���A]]])
����&�u�����/4h�������;iii�����WFF����3
!88X__��A���1bD���j��~/^��9���WZZ���#������744�{��t|������K�:::�[�����7o��k��166~�!L�2�?����C����U*����+RTR�FFF�g�|b���]m����k����j��U��W���&M;v����)���Y�f�����j>��};//��Z�������qii��S��K��0.����O�>}����\�V�mmm����9���WRR"_��z�~���3����
!BBB�l�R\\|������O�2�������y��.GDD��[ZZ���}��w999����r�J{�g�WW�����\�P(|||,X����fee8p`��9�������)��������������;w����4(???""b��Q���>>>�g�.��I�&]�|�C����={������v��u���C�)((����?��;����y����|��-++�~��i^Z,S�D�����G�Wrlj�Z��s������K�@�p����;J�%��j��Z-��tttRRR^���SSS


���T*�?�p����}��px%�����m�~��!!!u���0a��E��w��|x%���c`���t�t�~��#�5^MJ�*����
^\XX����+d��������QPsU��oPP����T*�\��w�
6�T*




,,,j��������Tm����^^^<��+F��t���y�����
���c��y�O� iii*���Z{�����G�N�z���������j�����^-���'4K���9::�E����vvv,P�������G���
6477

*--�`cbb�������������S��/n�����a���,X�i������[s����f����������acc������'M�0�K�.yyy��gffj�d���
�">>��=wqq��������C�zj��~�������������DDD������&%%EEE-Y����������dgg;::����"�X�jUhh�������6m��z�����KE���3f����}��A\\��9s���[n�J�2!!���%--���S����/��e���Y������o��T7333##c���<��+��7|NJJJJJ*s���[{��3�����-��-[������)�h��MpppLLLPP�|����������B��'~��W�FTT���_�~�����OLL����4E�F������s��c���N��������Oaff�������gEWWOOO<}��^�zedd���V0��k��Y^aU����,o��g��Ujy��7��r�����p����'O�r�J�8gee9::j>:99�{#�Z}���-Zh�t��Y�y��K����4E������yS��������P��s����[�jebb�@��U�B)�+����z_GG�;vh>�[���h��������=�Z]RR����S��w�����"a�Z��rd�RY�V��/.��~�"T��jW����>��k�n���3��Y�f������o��S�����|-[[�+W�H�j����������P(�����9�9�����}��	
����Y�W����NNNRZ������[��k����S�"C��
��?++����������Y�y������������P�V�T*�XZ>����e��������w��}��)R���	!4��e������aCjj�������sss+�����u�����;???--m�����_������9�V�SRR���4�n���������O!�w�}���V�Q[ZZ���}��w999������%{��quu}���OSPP�������R�JJJ�c���j4���E����K������.�8x��[o�5h��������Q�F��������=[�R{�������!Dzz��������]�6�� IDAT���gO!��	p���r�1r��]�v���v�����M�����n��5n��7n�����;w���R����>��U+ss���zk���.\����v�:r��!C�X[[��?_�������_�^�\�n]!���;�����Errr�=T+�������#�Z��>�6<�tttRRRt��t�~��#�5�_��H�@�G�j�o����P��+V�(y��zER�[�lqww700h��QHHH~~~�U�JeAAAAA�������N�<���������i��Y*��y|��������+0�������Y����kYYY[�n-��R��	(��a�._�������>n�����y��=O�/-�/��@5�*�����x��7�,Y���������(��,�]�xq��M


�7o�`�����r�����������%�8z����S``������3����4i2l�0�����?>44���������'//��{��***�8q������A�-�l�"����"::�E����vvv,P���w��y�B�����Hl/_��s���k�v�����~�������g�����777###{{���0�5��x���<�������������}�j�t��q������B����3f���>x� ..n��9s��������233���7n�ill|�������W�
!�JeBB���KZZ��S�>���_���L�y��}��7?����{�}}}��w#&&&"""666777)))**j��%����}AAA�v�4g��?omm-�Z�*44t���YYY�6mZ�z�������y�������sRRRRRR������+A��}��Z����zbiTT��Q��u�&�������c�����R�z��URRRTT��k��M�*������+
{{{???!������gzzz�
���4i��~(�w��Q~�azzz������l�2OOO!D�6m���cbb���4w���WFF���mE����������d���<x��&�~~~���B�o�>11�����>���<�*�^�~����}�B�2{k\�t���M����=33S{S�J077���B��WO�P���<z�H*����\fhhX�
�ej���~��g�7n��a�F������v���������M�<���+��-�j������F]TT��O����?~\
�Z�>�|�-4�t��Y���ZE�������!)�+��-7�+�����]���7�O��j�BQ�z�RY�V�g�R���>�fE�d<������G�:doo_RR"e=��edd4{�����������3g��={�n�����k�~b���E���^%��]��Z��X�P�o���o����t�������g
�������'N�prr*7Ak``������k/?*����7n���������Y���T����o�Y{�����O��w��������:U(NNNg��������+��j��;�bt�g�*����������?>55u���^^^o����q```||��#G�juJJJ\\�����B�T*�XZ���Y��������B����.--��G@WW���&##Cq������z����{���!!!���[�l)..>�|����L��}��={\]]���;w�<�-^�n����������F�y��u��ZE���^%����t��g�����{zz�������s�������n��5n��7n�����;w���R���`I@@@@@�"==���}����n������[o�5z��.��q���XYY999�X����NJ��m�V���A����#""F�enn���3{��������U*����O�9���AAA��]���;v��i��a���y�U�HNN�����j���c;v,--B��j�iW]�@GG'%%E�x@MG�j<R�P������/�x�~��#�[~~~���O,*,,T�m���
�K��N�.[�L�$.�t�J���������������W_999���;88DGGk���9�L��O�.�m�V�P��U�a�����������+V�h��U�:uZ�l���>����#F\�r���+�w�B���C�8n���iV�T*����dlll@@�������?���O>Y�d�TT���~������C��?�a�!����w�}�<���m[HH��3��??{��O?�t��m|�j�_���[�n�&M�4ibmm-�����>��WOQTT4q�D[[[�-Zl��E��B��5kV��
�����SPPP�;FGG�h�������n��j���*���>>>S�Lqss���O}}}���_ZZ*�011)�y��999u��}��������
!6o��P(���+����1c�|�����3f�&
���~�j��y�|��?�p�����@__��W�jJ�9RTTt������;v�����`LLLDDDlllnnnRRRTTT���������^�zi�����������3�e���������\(�j����;u��9��c��~�������H�>��I�N�8������?j�������tM�R�711i����#�o�^n���-���������i��Mppp�o��u�����Rs���Jq���gK�F������3�z��������H�����i�
���=x����t	�����~��g,((���BjJ]]]k��%;::VdA���������5g���JKK���H___!m�,������Dyyyej?kL�����Msss�
����X��T��������^/+������J�������,Y�����#�$��������3�z_�|�]�rE��I����n���o����{jj���WNN�R��^��G���T*��?�<n�8{{{!��'�\p��E��������5k�,55U������b\���o����]�4g�����I���?�pV�^��m��
j�����������Hu�B��]�#G�h�:t�C�����B���tuumll222���]����W����w�R�Z����t�R�Ju������z���RQ���*�J:��G		ILL��eKqq�����w�>e��r{2u��o��6""�����-��a��i���{��effJk�o��������}����+W<8x���G�.\�P��={����&%%U0 �|�I||���k���V�Z�n��������]�O`ee����b�
;;;)S;q��������_�v������d��A�|��T���@�B@@@@@�"==���}��A����F�277����={v��=ztQQ����g����Q�e���3F*����5k�t��{�	!�M�!�IHHHHH�[�n�v�~�����;?O4����x�����U�Vyyy����"99�G���c��u�����T�V���������6|���/�x�~��#�5�_��H�@�G��2
[�b�?��BY��>>>>>>�)P���[�lqww700h��QHHH~~~�U�JeAAAAA���EuB\\\���k\�W�X��U�:u��l�2&&�G�4�������f��k��deem�����J����b��15.���m		Y�n]�~������7h������;	T��}�oDD��o��d�oo���D!�f������6mjhh��y��������������G///!���G���mllf����I�a��I�
<x������666>>>yyy������}��l�,���G����6lhnnTfP���-Z�044���[�`�Z�B�T���;kz+�0`�[o�URR�9�y�f�B_��GGG�3��>���8p��1c�,Y���u�7///--�o���3;v��u����">>~�����<����3g���s+}/==���Lww��7FFF8p !!����B�R���������v�������������&�;���\�H�����o������������v���������������MJJ����������7o��woBB�b�������iS�Z�*
�Z}���N�:iG���~��%��u�����j�������QQQ�F�������n������z�����J�������cQQQ��]�6m�P(����
�������Oaff������^n�
�B�T*�J�B�x��LLL���uuu����8qb���5��-[�����������M���`�+xmmm��[7q������l���L�z��������Y������4k�������<x�w��u����@!�i��K�.���i>���gff�T�����a���������}8�$PD'��R}�DI�W�������	1�JM�PT��)��7E����@9���)�	�b�(�t`�������N�y��k��]�^���������Z�BXXXH��R�����P��u�LMM�r�s�n�aAAA���u!WWW�unnnXX�������N�3f�����~��W^ye������h4={������I�i��o��666�_��6��x�S�vvv�Z�*,,�/���B��|s����N�8��%�R��=�y�eY�?WY���033�����TWW�W8{�l���O�<y�s377711)**���������)..666����7	��G:�+I���?��c]���gmmmO�:%IR�.]�O]���vuum4AkbbRYY���p�Bs{jGG���<�Z����l]�k���������"������?q������������u[ZZz������&���<�+����A������Q�����eff��5+33���?���y����w�.�NHH8x��,�������!!!J���Z�V�V�,�����#G~��W!��}���������tC��Qo������m�������������2](44t��]III555g�����Z�h�����x�����:l��m��%���6%%���=99����7o^BB����_PP��{�m��544�$pw�����CSRR�;���������������,Y2s�L�F3s��U�VM�>]	��PTT�\;vL�����gOww�������Fs�w���V:77��7�P�������|||F���S����g�}V7C__������p++���G{{{�[�NQZZ:q����8P1l��y��M�8��{|oi���qqqQQQ��u{����{�=~����RSSG��B@����1x�`e'�,�������J�JOO����������/�x�~��#�-�_h�H�@��H�~ccc�[Y�v��?�������FFF�:u�����/_����^�u%��_?I���m;~��o������q�����������,]����V)�e9""B��������ZYY��h&����i�������>��3!��={���3g��3���


8|���/�<o����%Ro��������I���9�m�6!�����}��e[�n�;w�o�����f�%�j�����
6���O			[�n]�t)���x�S�������������B���+��������U.**244���O�~~~S�Ly��7�������N�ZUU���l�urr����e���DDD���-Z��w�������)SV�^]WW'�����7CKKK�Gpuu������O����B���S�����&.Hnn��^x�GG����o&L���/��������������8j��Q�Fegg���	��{k�F�rss��i��d��-NNNc��B�������;w���_|����+�j7n���+++KNN�������������|ooo]�3�<s���s������M�v�������~��+W�X�Bw{��9{{{���W^����.��j��������'O�=�	R��&IRpp��������,���{/���J�R�FFFaaa��_�~���~���066v������*��o��s����qc�c����B;;;]��p�|����}��B�K�.�J���srr��w+��[oeee-_�\�p���FFF����W��6m�	�L�����S�������[���v�����]�[�n�$)��:u������
�nX�`A^^�rt��	!��(gD+i��+//��J�����S�����ey��%����]]]�C�-����/_>w����8^�� �{[������������w��4iR�6mnYS�e]���,&&F�S]]�p���QQXX�+Q��*�M���'�pvv���������KJJ���x����=���m�>}BCCW�^=���S��4�~2g�����������9s����?�{��9��e!D��]333u���"e3n��i����w�^]IJJ���s��h��7o���_��mu%���'N�())ib555&L�������{���+�e���q����CC���Z��� ��GG�	&L�4i�������������S�Nm�����W)

��kWRRRMM��3g���-Z��@K�,��������;�n��m�����kJ���$??_�|�����|�T��k���������':th�����������'''7�yW�Z��w����������������T!I���_dd��\PP����r���������5K������;wn���^��#.^�hoo_QQ���������^�~=<<|��666~~~+V�ht�����������X��}�����/������~��7�����oB��^{-<<\)ILLLLL4770`��_=d��{y�w�y���+�;w��������	!�z��6m���;������������4Rjj��Q�X��IJJ


���W��H����_�rE��f��222\WW'��eY�B�e!�r�R������{[�,ggg��7o����y_hn�h����#�=:k����@V@sF���������X���%����/�x�~��#�{7*++�6m���O�����������III&&&���

�~�z�M���+*****lmm������(//����3�����:urvv�-���~���'N�8l�����
6���{��iMihlllll������FVV�V��eH��1������w~�@���oxx��a�bbb���3n��]�v	!t��_���bjj��[�������F;���7n���r��C�\]]����/_�t�Rgg���'+�&N�8k�����������������-::�{�����NNN����,7:Vuuu``�F����y����������^z���C������������ehhaoooii9e��z;�w��)IRBB���?�����~:w�\~�@���oyyyVV���cu%���{����"!!a��eqqqW�^���_�r��U��z,CC���|����GDDh4����������!����������?~�����~�^m������qqqeee����� IDAT��QQQ111J���bcc?���/������j�z��������-[���*n����n����_�����_~���={�q�B���������8sss~�@���oQQ�,������FEE��1c���j�z��!����7o����Z���w����������"I��K���$u�����_amm���y���{+66v������*��o��s����q�j`���w���0���x����iW�Z����o�?����|��5�=�X���G�����?oo�������Z�l������#��M�H�~MLL�����9{�l���u�������m�����j!����$I*����Z	u��QW����|���
���`����<��������...��f�T�z�j�p��u�e�5M��k4��={ZZZ6�������w�]���'�D�t�����U�V�����BY��w�*���
�h�z����������YLL�����Z�R5<V]]]�V��b�;]����l�������'�D�t�W���~������g�����:uJ��.]�����������h�������Rw{�������v������-**j�6b;;���|������������&SZZz�������T>|�pNN��I�llllll^�������~����������5kVff��������SOu��]���p��AY�������CBB�V�7!�Z�r���k�#G�����B�}���������4BCCw�����TSSs��//�E�5���1cv�����?������������������������yyyM�gJJ���{rrrS*�������7��?�]�v��������
�����:thJJ��c�<==���������%Kf����hf���j�����+!���������c��	!<=={��������P������i���������[YY�=���{��u�v8��#F4������`����'9>|���/�������������C�j��Q�F�100ptt422��
�����:j�(
��?��3�:uz���Y�����1x�`e��,���a���J�JOOW�^����455MNN�j�_�uZZ���cY��#��?���o��5������/����u����X��S���:-�~��#�-�_h�H�><�������J�qs�c��
>���R�RY[[_�v�)�4O�t�766V���k�>4���3��\>u�����fee���5�U3QQQ�������P����6m��><u���1c������.�h���C��/�����Z�����w����w�[5]]]�J�@�W��k�:�d�����:����_sssggggg�v��	!����[���T�����*~���B??�)S��������fffS�N�����������������Sdd�,�
O���C��������/_�t�������u��_���bjj��[������:����z���m�����			��!���u����llllhhXWW7a��B���m��sH�t���e�����I������p����$%$$������m���AAAc��������[5j�����M�t%[�lqrr3f����(55������s���/V�\�T��qcxxx\\\YYYrrrTTTLLL�c���{xxl��=""B�����%&&*�����e�����]�z5>>~����V�RFEE}��Giii�.]������{u}����������K�.��+--�r��b��}J�S�N5����rrr�h4!!!�>p��o�����.���999���>R��&IRpp��������,���{/���r��$IFFFaaa��_�~���~���066v������*��o��s����qc��i�Zoo���;WUU
>���E��K�.	!���f��1r�H�Z=d�������7+�v��5e�www##�9s���,I�������$I��222222B*uZ�j�h�������������nnnVVVvvv�m���srr��w��T�#�{[S�N�����{�"--������M�u����v�����@����
���`����<���o���F�V!,,,$IR�T���B��g������#??_��
!


tg5!\]]�j4�\j�z���JF�f��g�����Mn��E��BBBx�����-SS����w�yG����N�4�M�6��)��.
lff#����V�
7�����z�166600P�����-U����s����������s��E�x���Y���3g���iii������;wN��}���u��]333u������[�����$I������Gu%������J>���1//O)�e9;;{���h)���JKK

����������7h"v�6���q��	�&M0`����~���(**������S[�n���U�CCCw�����TSSs��//�{�����p��AY�������u �?~��m������aaaeee�VUUU�������,k�Z���s�hu/����������|����������2�V[[[�\�o}M�~\\\T�|��/^����������uyP__������p++���G{{{�[��^&����d���3gj4��3g�Z�j���Jh��>>>�G����Suu���>�K����������������������V����N����������o�����\������'����5������������|���������\��w�^�����1x�`e��,���,!��J�����~oK�z��y��.]��������m�9�����f�
d54g�~o�����](!!���|�Xh�H�@�G�Z<R��3|������?��C�ggg����������QO����H�����_2���rKK���Ull�t+k���KfX^^>i���^{������6m���g���{���q�F���������N�ZVV��h��%���������$�.:���S��)������j��M�6���Gq���1c�������]��M��d�����������/���CCC�n�:h�����~����=��s�=���a����~{�����@�8�Y:�����.�~�zSS�n��EFF���)��$�>}z��evvv�$���
!:�������|���K�:;;O�<YiUUU5g�GGG����'%%�z����Z^^�������?��;wJ������'277W��]�vB{{{����B���{wSSS''���HY��Z�v��!��
!�}���������m�oG��
6��3��v����_|����wtt�0a��/��_���h��9111��4���m%$$,[�,..�������+W�\�j�rrr�h4!!!�>p��o��+�044��������}{DD�F�IKKKLL<��������������������)S�(�B����-[����U������6n�WVV�����\�j���;?�����D!����322v��a``p/3<v�Xaa����~�,�G�y��'u%�����4��3�<STT�����4���mEEE��1c���j�z��!����7o�E���N�>������feeegg��k�Zoo���;WUU
>���E��K�.	!��������K##�3f���=zTi�V�


��7�;w���;''g��q��\����g����T�T}���;w��;����[�n�3g�7�|�m�6{{�&��v�?�R�z���_x�����Jkkk�nfkk�����W��W����������y�����N�2�3l�0������{��U�������}�V�V�;�����mll�j����B�$�JU]]-�(++[�pa���m��}{!Deee'��hz��iiiy������;�y��yyy�����3}����~��W^9r��w��KK��}�M�6}�����QNu��N�$������+��@��,A��������511B��|��Vccc��c�nGl=��;n��������w�����VI�����V�X|�
g��m�����'���7/������IQQ����������o766�}�X��M��_��U��z�O�J���K���B���lWW�&�r|3�V��w���9�c��Jo�*4�sii��'JJJ��y�v������-***//���_������������j�o�������u��u]
0���������4���`Y���W�R�a�~o+888!!�����,����������uoj����!''Gq�����������������W\\|�����<��)))��������\����v�JJJ���9s������E���?���o$&&v��a��mK�,9|�pgx;�{������{����KHHx���

�{���[������s������F�a�|[���?�����3W�Z5}��{�0>>>((������u��MNNNJ�u��9B����O�>��^���h��������������������g��acc����b�
!Dii���/^<p�@!��a����7q���G�*���������!%%�^w���qqq���

����{�����S���}����H���RSSG��B��Y�f��
~��'##������vqqy��W/^�����1x�`����,�_��,�P.T*Uzz:>���={�Z�^�~������$I�
�a���u�����O?����#�����&��[�l�g�}fnn��"��?�����_�����{�����#������]�%%%,"����-�_h��g�w���VVVVVV666�{�

-..��>�_���C�?a!���-Z��W��m�v��988���T}������okk��O���x]yzz������gy������___��'O9rd�������'O���;��j������F����|��W����O����<�������/|��WS�N]�p�����������_33�v��u��a���+W�����p����?����aoo?`��
6��c�������]�v��w��7u�����g�y����V7�z�����?���}�N�:���/\����W	]�t���*77����rss����>}�n��7>�����=z���a�,�JyFF����x�	kk�A�m����'�TB������CCC���CBB&M��n�:]C��=�o��T*�����o��VTTTAA��
��[�~�z��7�|�n��;v~���_}���~����Ykjjz�'''%�g��O>�����j����>244TB�/>r����{���"##���-[��]}���������;�6mRB]�v���OtG7:t��YB�����~�i����1b��K�tIh���oY������\�p��q���kb����aaapttB���;S�L>|�b�����M���^}�U!�����j����������CLN�V����7�p���}��B,X�`����Q[[�����#FFF�����n@@��A���g���e��/�,����~��W���:t����{n���B���B!������v��	!~���?�#���[���9�����|���`gg7p�@���_J�O?���gO]5ww�.(;�3�S�N����������x�bS�]XX����T�=z�G
F�]/��L#22Rw|��e���?�=��}�O?��������?���}||BCC�������N���%I���L���g�\o������?t��w�}w���������
!dY�9Qjdd�dU5MJJ�_|1l�����>}�|��g�TWW��U�*����\hjj�����b�Hj����������EGG����g��U���+�:��R�AC������YXX���8;;����
%I�������u%999�:uR�����eee�{��;wnjj��)S�}�]������V�������'O6ez]�t9z����������B�'N����B�?�������=����}FF�.�����m�V9��U��;��6`��Y������dY�����n��|aW�t�R??�_~�EQ\\|��www]��m�^�~��/����_rss��?��{yy}���g�����Y�v���m����?���O>���������>�������:u:���i����������#������K��=U*����w�������=�c��i��)[�����('H��F��3i����������/�k�n����'OVB�/^�x��!C�]�fii��������zzzN�<9  ������n����'~������������_|��#F���7:����WTT�]�v��9���&LX�x����d����V�����r��F�y���7l��|-x��y�O�>|���]qq����,X��m�����_kkk�-)55u��Qw���������'�����;�Y���
�����s����k��-�7��'O<X9uX�e�Y����J�JOOW���<z�����g�}��j<x�����G?����g����������������W
@3�RS��?���+^{�5GG���������z�A:}������K���������@3��|���#q�3@G}��---YD�k��Z<R���5��oll�t+k��m>+��_?I���m;~��o����
���9��,R���M�����������{��Qng����k��Ig����m����s��}Mi�����j�����x��G\�H����;;;;;;�k�Naoo��ZXX����?^W��������O?B���M�2��7����633�:ujUU��fttt���MMM���"##eY�t����$%$$��<]]]���>����������JyUU��9sMLL�w�����k��K/
:���\��9??��V
(((�$������-����$���W]�~������i�n�"##���������_���K�>������Cs����Q����6m��+��e�����1c�FFF���555���;p��_|�r�J�������������������bbb����M�v�����!�[o���G}���%%%���S�L���ccc�l�bffVqC��P������hBBB>|����~�-66V	%$$,[�,..�������+W�\�j��G�?��c�~N�:����x84���$I����7oV������{�����*�J����i4�~��������J������g{zz�T��}���;w����={{{����7��&��}{!��K����������������3jjj�=�TS�����B�$IRB
�j������������������������3f�9R�V2$00p���B/���7�xxx|���B��'O�����pP5�)N�:���v���B������/����u�%S;u��l�B��������Y^�`A^^��gq5M��=---�nV���B%]VV�p��:�m�VI	WVV6����B����c��+?{�l���u����Z��G�����D$� IDAT�����c���uRRRuu���_v����555
|��w������I���is���,���fff111����j%S{_���	!������������qq���)��������g�166600����V��������������?j�Zv�
U����9s233����������:w�����}V��k���L]���H���SZZz�����������������m[�V��w���9�c��B����z5o��
!mu�$I��������������$Y[[�������C����...�v��������<ZF����q��	�&M0`����~���(**������S[�n���U�CCCw�����TSSs��//�E��7LIIqwwONNn�4�]������7�L�8���Ck��B��j���!��""",,,�\��keggW^^�o�������O+��mu���<(�rzzz|||HH�����|�|�x�����mc�/�0Q������s��
��G�q��E{{���
___]~����������3f�������[�b�=�!11111���|��_���!C������� {{{WW�M�6999)�P6(>|���/��BEEE�v�V�^���m��]�������g��YXX����j�����+�=z|�������E��}�Y�~��!����5��O4)))444??_��#]�������{��������<xp]]�B�e�Y����J�JOOo�~eY����7o���K���E���9�����f�
�7k������v������b	��#�-�_h�H�����>>>������~���/I��3g������---������pww�eR��4�#��7++K���2$I�}���k���4hPaaa�CpGZF�7::�{�����NNN����,+�'N�5k�������������W^^�����5�������<f�%��K/
:���\�!??_7���aDD��������)S�_��?��;wJ����������-\�0((h���M�i���7���������%''GEE���(!cc���D77�������8p����VB���}���_~y��E�Z�c�CCC]h��-fff7t��A7\FF����z���/��r��=��N~~~NNN@@�������oll����===U*U��}����q�F%$IR������������G�UB�w����0`������K���u��j%
l|��!����k��y������?b����,��x{{����7���o���]������sss������t%���uuu*�J��cG]�����+W���/�?^�V�T�z�j�p��u�e�5��C]�F�������]�fff111����j%�+�������������ZhYZ@��k�������������F[������+�uuu�����.a�����'N�����h>�W����K�z��������v�JJJ���9s������E��j��1;w����kjj����e����������W\\|������&�0%%���=99���+**�������Zmmm�r][[�p�H�����O?���/�������^�~=<<|��666~~~+V�h��������O�
211�={�������t����O�>��^���h��������|��������[sss!���~����@����RSSG��P>�,�����y��N�:��������VK�eY�B�e!�r�R����U�*dff���&''k�����:--m����Jm��_�~k��	

577�����[������CI�?[PPPPPc=K-�Z���oYhVb�Aev�@�����{t��
�.��RS*�R���������}x��v5#|�!���m�B5�����������T��=��~��/~:�sY;���=��b_`����11T}1��g������Z�m����U�����&�h����+gxJw+�T����;��Z[�r�0�l[�����Y+��,�<�k����*�{�!��R�����[e%���������|���~B/�vF%�G/���B���
���;>���JH���E%�:�I3|�������,������/q�^����������������U�'�?�,T"+Fw672����j+Fw�4Q+�nW�t�V��Z��limj�u��������N��H��z�
!���4���8���D�q]���='i���gx�Th�g]��`ai���2- �[\���4�w����m�p��c���nm�6��������p�S����fV����W>����^�����i�wo�������x������uQ��
�.!������t���(��������Z�����p���u��lV��m8tAc��h�����Jb;"-�����Ou��w~P*����g��jje!D^z��s�S��'��8X�P�s������P�qt�,������}����.^B�.*�v�����M�z�Yv���+��;��<���)������������"����{s-�����s��;_*�8z���9E���6���eq��J��[Z���t�k[3%T['kkeq�3�3�~~���k�M\C%��au�S�J�W?���������!O�nO\.�u��&��:t��h��Fp�37*��Bw{���F�&	���$1�����K�t�����C?�O���@%�6v�ta�g8W��5�(�����V]���x >�g�������$���V5������w�S�#��MO�@%�no�L|K��:�6���A����Vy_�����C@���~u�����\SW��[����[E[3����������Fr�EeU�4���$�[[����n"q���������'K���B�������pkkv��RI�Vi���u!;����My4Y4�nmd`gnTt��Z������u}/�����@�J�j�VI��VI���}v��B/7/7k�Jrnc����������[	!t��=�_��mL7����[r454h�4��<��c�.V���n�fo��lkn��v��o=��h!	��������7�7��[Ew���bHG��z��])�11T
�h����c��~���V���������U����*�@%)�*I�<���i/�km�����O:�~�I�t2���?�j�V�8�q���%5�g�l�O�R�������Ou\�TG!���,.O�������	�����������u���b�V��;���U}�S��w������f�o�ty�u������/��D�J�)��h��4��+j2�&�;�1i�Sg���Dq����Z�KYu�����>|��Oq�d������B�|r�����
)55u�	����u����B���!��B�P�T���*�Z:R��������/�xj!��y�XhV222�^�]���=�����J��M�6=�����������PI�7))�������}�������_o����qEEEEE����_8�?y��������UTT����T��!����<q��a��eddl��a�����MkJCccccc��|��4����Z�-C�$����]��o��A��3��!�>l�����>}��7n��]B������������v��-22�������������[=t����kpp��������.]���<y�d����g���x��>}�888���������EGGw��������)22R��F�����h4666���zpp��1c��K/�4t����r�������

#""���---�L�Ro����;%IJHHh�����.\4v�X~f����S����YYY�����������T����l������W�����\�r��Uw=���a~~�������#""4MZZZbb�����������nnnYY���;��������0���pDDM�MM,�@��F�� )FBH9��h����@��MR��)9x�.
�3W������$�9�?��s��'�����g��Z{��=z<���O�9s���#����m��U111k��)++KII���]�b���Yqqq[�n��o��k�
�������4�����?888hw����~����3�����c�Cv>777+++88��1�	���oQQ�Z�V*�������?��7�P(����0a��u��q����������s������k'���_�.���dNNN���BKKK�����yV\\\hh����\.�����)SV�Z%��yVrr���_���


���LMM5
*
)
l��C�MMM/^�������k��A�N������wVV��f�������<1:��522BhNB�Kvvv�n�4?�w��������6����B�B�l�R&������j)���������!|�|�rtt��p�i�����h�~���]���]������]�vm��:u��������y33�.]�����74C:��������/,,�.���B��j�]�CC�-Z��G��y�{�lL�	LLLV�X��R]]-����J�j���?;-��O��d�>}�l��MS���mmm}��y�L��C�S�333]\\L�UUUi~���?��u��1==]�����1��mllrss�k�Ju������i��y����gKKK���!�30�������������~�������^����"<<|��
G�U��iii���R��?!jkk�k����;�<y���~B�������>Z�t#22r��-III555/^����1cF�
2$11���K555111we�mll���w��]\\|������F�s��=nnn)))�����������������N�����Ox��������s��i���`oo�M�6I��������'N433�8q��������������������O!���=<<�t�����g�������|~��������5kbbb,,,����l���:u��A���������E���k��<p�����Q�F���y{{�����i\�z�255555���oSSS��={��'<���T///&�Y����7�|������?gZ�r���~��I�T�j���t��t!��������3&==���8%%�������:t(�<�H�>k���/^ijj:i��e��yzz2-��M�<{����������]���H���#�:��/��g!����.��Z�h��u����������X^^nnn�z�V�.\���b``�T*?�����*�*++���|||X�����;z���/����Bxxx����1�N�:U[[��7������+W^�re��
7n����B�o���{w��}Y���I�������xzzn��}�����T~�����'���u��9))ISe��I(//��!77��Z���D�L�a��Fv���|������vvv^^^^^^���R���u���aaaC�e9h�g�[�����p�BAA�b��[�n=x�`iiixxx@@@^^�t[\\\BB���I��P=����>6l����655555u��1R(777+++88�������o��m���_BL�:533�C�����������nS(zzzB�?�d2)TO-���wVV��������)S���-Z4n�8�P__�U�!)��!���!�r����l��������J���l��Zffffff�c3f�x��w><e���������G�L����!��������vrr���S(5�����Z����������������S'L� ���!=���u������n�������'NtrrBdff�u���gm
�B��y����������Z����[�h��DOO������QyFR��o����9t��;��s����K�
!
���mVV�"??���-[�,))�����)//��{wqq�����
�B������-%%�1}��d���K�,��m[AA�����7|�pi�oeeeYYYYYYmmm]]�tMb������i��M�6����������������������J����������g��!��<y�b���AAA�F����l����E������j��j�j��)7n�P*���
����B�z�:w���NSSS!���;}||X�I�������D@�r���~���T*!�Z���P��B�B.������/�u�~@����G�t��!����2��t��t�_�y�2�;p�@++�n��EFF?d�O`"���g����k���[�o�><<���������{��emm��G���xMyZZ���Evv6+	�S��w�����;w����K�.MKK3f�C6x�������lD�R5xODD�������/\���7�=z4,,L
m�����>5j������;}������t4�8�kbb��M�7�xc��y������R��/�����R�������+���_|�E�=��i��s�O?�T�����|��7+**,��i�����?l�����sLL������������-,,._��`�WWW��� ��V�Z��K/)��^xa���j�Z*?~��;����K/YZZ���w���/��������~����H77������G/[�LS�����_�\����M�,XP�r��e��}��g�m�Z�l�������������N
-^�8..�����?���K����;vl�����s
�b���zzzR����e��3g�<y���]�rrr�,Y"��.]�|��������_~������P���o��9�y��!!!B����+W�4H3�A�]�~]����Nq���������
}}}}}}��������{��mgg'����/(������q������?�X��/�(
!�����c���9�BJ����������S1m����D������K�N�<i``�]�v������}�
!�w��������!�/_��������e��#G��������P�T*5��i�Fq���'�bh��w����md���o���������������_-�_�r�K�.����������������������|��k��5��������\.����-Z�<x�]y_�K�,�={����<����m�������3~�aqq���Odd������N��t��L&c�h&x��v����}�
v���'N���}������B�Z}o����@��������g������Jjjj�=�����R�����������Bcc�������X:�Zboo?f�����z�����������PRR��G����9��[��Z�\��-��k���hjj�)��d���?s���$++���Y�������u��m��)���k����{�Y[[k���R���;���u��!##C�������Bq��Y���bM��_B�H�� IDAT��u����W*�����~�����[KY@s �?�����		��i����/�Z}����7J_�BDEE�������B�����/���i*�n����b��}�����������rOO�m��egg���,]�T��6(44t�����o������~������#�pvv���7n�O?�t����'O~��'�������\.3fLrr��������y��q��-�����|-�	���(��cF�]\\��/��i�&**j��1Rh���3g�����������}||f�������1f�������*�9s�H��
������444|���
T^^�`7�^YY�t����'[ZZ�1b���B##�]�v��??00���������__�r�����>�������)..>|��i����+;���������������zyy5�~ii�����'��~�m''���?������9�k��			����s����'�:�V��/�j�B����iiirdFF�R�����kkk�=z�����?�':::������s��-���UUU,5����~_|���s�~��'vvv���111�����~hPPPXXXTT�������Y=�	>��a�����onn�$����_�y�~�Y��������KUUU�?�^���,��.\�K�_V�H������g����j�����w�ndd��m������
��bhhXYYYYYimm��*�����x||��o���/K�:1n�8!������c�777!D�V���L������;���			���aaa�����?����	�N�:U[[[�
���s���f������������M�6B�R)�l��ejjj�-���47������S�������ZZZ����;���;�;�/_��sgccc{{�%K���j��&&&�d�
64��111�����+z������e�!�f��g�}��];cc�N�:-Y�D�R	!�;���nkk;g����(GG�1c��B��sg���vvvFFF�;wNJJ�t��w�			�9sf�=lmm�������Puuuhhh������"""�>4��U��u�fbb�����d���j����A�k��I(//�l����B�����{|�o���C@G5�o�zyy���j6!!���~��!B��������W�9rd�������n[�jULL��5k���RRRbccW�X��n����:uj�����~��%''!6l�0{��5k���u+>>~��y���B������v����o�]�p�����6m����WOH�`���[�<x���4<<<  @��644��i�����S���9s����?�\
���n�������_o����]�3����j���s��-((��y��u�f��%��0��_J=�y_qqq			&&&�ppp�B��u�J:t�pW��|�s�������d�������v��������������R��� ::������=00�����*������zxx����={N�2e��U�-{{{gee���6�EEEj�Z�T�7;~��7�xC�P�����	����B����������s������k'���_�^h������:t000?~|MMMFF�fB����������-[���&O�����/66600p��a�Z������-[^~���W=/���/�B���'�0��L&���]%�y)��!������c��������{���_~�%88X����&����\PP ]_�|9::Zs��i�rrr��@633������yc�`dd$����h����n��i~v��=77Wsl����B�B�l�R&��������Ceee��Owpph��u��m�UUU�����4�������

��o�	���48.�Z}�����;kJ��������q=�������g����!��t �kll<a��/��R�v����G�j���w��jM����d��j-����^�&������/,,�.���������-Z�����j<(������u�����bi+mcj����������
V__��)��W�������G�y@W�u���'ONOO?p�@JJ�����CW�^�����
!:v�����	�������g��---mLd2Y�>}�m��)������>��L&����������...MN.����8qb���������FV���������juc*�d2�s��iJ���/}Y�a����R�������G�R�fN7R�vvv#F�=zt����w��***������9������������-[�$%%���\�x���s��������������>DGG������������>>>����tTrxx��
�=�V�������#""�<X�Bakk���%����_�pa��-KJJ�8|��������7o����.++k���L��q�����������SAAA��]�BM��^J������������eccS^^�{�����.h2�w���4�iP:&�����S�JG���W�\yW��A��]��T*+++���4�B??����������[YY�����;�a:0`��={���5������r�������B���7n��8qbaa�����������Y���aaaJ����e�����������|�i�����<X1i����z�1{����n�����ogg7a��O>��!����"}2Y&�������{��k���AAA�F����l����E��-����7o�����?����Cq���W_}����9����zyy5��&%%EFF���*�KW��������|�R�G��������N�U��������\.OKK��]��gk?�����(��/@���7�x####$$d��	�0���~�������
x��
/x��L�:R���H���#�����0tKsI�>|X&�]�x��<�������������������Z�^�p�������R��������X���/�����S����
�d�G����o����o�����w�����r��+W�l��a���QQQ�HM������w��������~��%j�Z*��wBBBf����G[[[���r)T]]=a�333++�Y�f���2D
M�4i������?���j�����p�B�RinnPQQ�����D�L�a��F����u���aaaC��+dnn�v�Zooo;;;/////���LV$�&�����U�bbb��YSVV����b�
)dhh�i�&WW�S�N�9s���#��������u��}��]��P(6o�����	%$$���T����A��������og����o��;4
6MnnnVVVpp���>�`��aB����������1c��"4��~���BCC=<<�ry��=�L��j�*)$���������R(99����w�����QQQ����
��6���!�����/~���{��5h��S�Niw���;++������������)S����-Z�h��q�HM�h�]�|�rttttt��DOOO�R��r!�����������D��v������k�\��k�F>�S�N�L�����AM����#��3�}����O�2���6$$�E	����]�&&&+V�Pk������B�
��T*U�;n��E�Olhj�����G�����-�:u�J�bQ��t ���c���t��������k������J�*����3��%�t����g����>���j�����E�4%zzzuuu�HM��R���_��"m������eKRRRMM���===g���`SC�ILL�t�RMMMLL�]�b������w_�p!''��=��g���[JJJ#����,+++++�����������d2�����%K�m�VPPp���y��
>\���y}����_������ZYY���UTT����?���������
65u��+W�������(44t���eee�������F�UYY��M�E�i3������s�4?MMM�;w����Y�`A�V��L�r��
�R9l�0��@��RSS��������j���o���������+��?��_?��d�Z�V���J%]K!!DZZ��Y����t���7���=z���,����d���-Z���MJ��M����/^�822r��Q�����-���dq��=�������Q=��T*�Ju��y�3<aaaaaa,�<9S���/�<R���H���k����8��,]��	������?ttt�������{����R)���~W���������e2����m��ZB��������m�622�����g�����������
�u(�C'��'�*��??d��;v���	!Z�j��{�����o�uuu�p���q�����n�*EG�=o�<��ry��������;����'$$������$''7y6n��}����������,n���Y�~MMMMMM��[�R�������s���>JII�������������'����?�����$==�I�&��u���+����nbb��Q���_���9r$$$���_Bxxx|��Wg�����v�?������y��WV�X!����G���ccc+**���z���ZZZFDD;v�����
�u���~,X�u������������I�����[�������k
�b���zzz����;'%%i�n~�����������O�:5t�PMI�~���������fnnnVVVpp0���������S333;t�```0~�������)��������woCC���(i��6oo���,__�F>k���VVV�:u����|������������4����***R��J��N���>�kR4�.���M�>���C�����r������k��.]����]��U���������rtt<t�P^^���w���������{�W���#G��3g��f++���y�ZFFFB�Z���t ����[]]}��a''���:��}V�T�c����CPPPPPPBBBpppDDD���������r�Z666�����������RJ���~�smm��'&N����$������������J�*����3wU�y����gKKK����O����EEE�www!��[���d2Y�>}�m��)������>�<�����S�
����6++K����p���-[���H�!C�$&&^�t���&&&������{��qssKIIi��\\\rrrF�y��������922�}����u�n�}�v��u��M����k�T*U��������CBB�������^{�s��M����������������:�����U<�t��������0�R����z�j{{�3f!&O�<u��+W�������(44t���eeeM~�������������������������+��M�6m��I���c����/]������_�U����j
0`��=�f�������9r����f�z��u��9�OSSS!���;}||X���M��������P��2�L�~��7���?��s�+]w��'''##�z�Q�T*�����r�jzz���qJJJmm���80t�PV���N����/^�822���t��I��-�������Q��������x����L�����[ZZ�$����_�y�~@�=�����-,,,,,����u�Y\\��mVTT888<��(//�1cF��][�n��}�����7oj����������u�=���5�iii����$O�#�����w����'O.]�4--m��1�����kkk��J��=�������p��7�|s�����0)�i���>�h��Q���;v������_���|<�����I�6m�x��y��������K�/���{��J��w��+W��N�~��=z�h��M���?��SM�722��7������������?��m�����111��O����B��_����|���\]]-,,���4�Z�j�K/��T*_x���+W��j���������K/�dii��o��+W����R���>{���###���"""F��l�2MEx���~�r����6mZ�`AlllAA���+�-[��g�I�:th��e�7o.,,��o�������;)�x����8cc�_�`oo/����w���}��s��)��[����I![[��-[��9�����v����Y�d�JHHX�t�����_�����~����W��B;v��}�������!����\�2h� ��
t��uM�:�}KSRRRRR�*������md��/_������������/�8p��O�>������o>��c!�/���P(,,,�������_�
)�khhxo���gO!��i�������.]:y�����v���k������+����{hhhBB����/�X�|�|��wo��9�O�>B���B!�R��4��M!��7��G��A���{o���y�o���������O�>���_��T~���.]�hnsss����v2d��������������]���~j2�r���^���h�b���w�}�n,Y�Ds|������������m�s��3g�|������>>>���RSB���:M#�Q�2�����x���������6l��c�N�8�������Z[[!�j���R)�jff�g���{����+���=z�����|�J�������{���.\�����b�Hj�����1c��������_�>;;[BII���Z*����o��jids-[�l����������P&��o����3����,ggg)\WWWVV��[�)S�����]�V��}dmm����J�:w�\c���C����������
!���g}||���5�_|Qq�����^�T?~\���[�n-d
���������#$$d��M�����j��'6n�(}aW�������
!���/^��������u����}�������/_�����===�m����]SS�t�R)���������o�����&;;�����3g����9//o��q?����7N�<��'��k��K�.r�|��1���R�:;;{��������,K
��H'H��x2�=ztqqqDD�/����M����1c�H��3g��9����o�677����5k�������1c������lll���#}�7<<<''����������4hPyyy��>|xee���K'O�lii9b���3g
!���v��5�������33��_}������?���.8������x�����M�n������KKK��'F������������O����~������/^�����;r���]�&$$�n<n'O�trr222���J�R���?/��Afdd(������������G�<x�c}���chh���;�l�RZZZUU�R�L�j���_�;w�'�|bgg��k�=�����EEE���>|��������v�����;�����P<d}sss&�.S�PYY���W�j����A������B$&&>d;�O��������u����Enn���)�����{�_�~*�J�V��D�Z�V�T��B���5�o������g�����w IDAT��gZ����^��r[[[����f�����R��9RXXx��)�$�g��~4��f��w��q>>>B����2d��nnnB�V�Z5��X�`�J��r�Jdd��a�233e2Y��amm���g����u���FT*�\�����f�
@'4�����������c�6m�J�R���e����-Z���in.**�����s����?  ��O?���411;v��;w4w._��s��������K�,�>�Z���(��6l���~�������m�v����-:s�LNNN���s�����������:w�����i���:44�u��VVV�l|�C��������J�1b��A�}����g�}��];cc�N�:-Y�D�����Lv�����g����d2??�����j����=����??u��w�}w��R(00���Wsg``�������&�����wUzyy����^�ZS���`oo?d�!���AjjjMM���W�9�w��y��I��Z�*&&f��5eee)))���+V�x��6�J	�z��`���[�<x���4<<<  @�������u���_���}�]�v5f���y�fII�b�������������B6l�={��5kn��?o�����K!{{{33���������#�������������c�����o��]OO��Z�L�����������������0q���'��[���~e2Yxx��u����j�����z������e2���Att������{``�w�}'U���

������={��2e��U��[����������xyyys��uww�������S�fffv�����`���555Rh��-nnn�'Ovpph���a````` ����344444����B��������7
E���'L��n�:ME�.������ZXX���48���/�����axx���cc����r�n4y6�g��a�����i�:t����%:�?;vl]]]rr�������Kpp�&��S'��v���

����/GGG��0m�����)�B33�.]����7�'���{������]\\���v���������O������u��m�
!����Z�2���4r�M����������������J?
���C��q#���;99i~���6�V�/�A�x�������y_��_cc�	&|���B��k��=�U�V��S�Vk��&&&+V�Pk���~���o��vfffLL�����e��Je��������:|�pqq������m]]����V�?����
CC�-Zh~j���6�����"�K;���K�o7�l:�o�7�+t"�+��<yrzz��RRR&O���z������V��c���tM������\����7��=[ZZ��n�l��}����������������Y���'N��8q��A633S�5;;����Z�V��g��L&������i�'...���������+�u]]�v����4;�������_J��!�~���������#F���w����k����bcckjj��?�q�F???�<22r��-III555/^����1c�v�={�������4�?���������O��B�������B���/\��e��%%%R��������LOO�y�ftttYYY#�����6l8z��Z�NKK������x�G���7����O�n��={�v�c��'O�������w�>u��&��Ki� �P�JG���W�\yW��A��]��T*+++���4�D??����������[YY�����;�Qu���f���#F�4h���W=����S*�...�W�����z8y��i�����<X1i����z��-�r��qc������vvv���

z��N����?d��L�������Ppp��#G�t�bee����_�t��_�#�
��#KMM���j�MJJ������U(���,))��k�3�n�;�����_����l���^{�5�J%�P���j�Z!]�����4��I����������"i�����
�~2�����]���2G��d���;�:M��%�������]��_�3EJ�VWW���
��H���S���O��4����^j��477�U�}���rft�_�y�J�^�zu���))){���r��c
5�����t���s�����V�jd�i��6�l<�kkkg�����lbb���2{�������***���5%�����k����m�B��������,@!RSS�WVVVBBBVVVaa���g���zu����������


���;cc����C4H���Af�i5eoo���������R9g�������-,,4%�f�z���nk���nVK��g���G���������_\jjj���m��:tH����C��oL���~_~����0��������7�A|�
2O�A//����kBc�����~Z����NNNs���N������o{�����={���0m���
�\ZZjii��iii����W�V?�P����'O������~����������'D�4�hd6�V�B??���w�<y���&33s���o���~)�������5k����j�����-))������m[ww��?�\�R=��gX�������:�X���\�JUSS��CM�������,--e2��
,--���s���zB4H���Af�i5(�?~���c������_|��w�}7((�	�����=<<�x��{�EZRR��?O�>}��EO��MV]]����WWW?c!��W_}e�%&&����u�F����oaaa`` m���d�/�4�Z���������
��A|�
2O���n������������u�����'�R�������.]zo�����t��[o���S������?^�b��Z6MSPP�u�����g,��C9rd��I�&=���������^�-������
�T*�����E�B��CM������QQQ���gzz���Oqq���a��-�	� 
>������b��eS�L�����������)SL�>�����^��1���^(..����'�D�����;j���bN
��m��UII��������U+i{�#5�L&������G5%�����\.�'D�4�hd6�V�B���:������4��y��������G[YYYYY��5����������B�	&���SssFF������^3Y�������{�u��������y��,55��z�?~��u��_?u���u���?��B����;������������v���`�i��6�l<���}GG��'O����={�C��&Mz�=�s�N��9s����TUU�����{���v�����]�������������n�������s�/i���RSSN�����III���OJJ���|��&�~�j�������LLL�v��q��F�h�m���Si���b�������������>}zyy��]����=�:u������~�����Y��(tH��=����;���_Yjj�,55���^j�[������
���?���u�~@����G�t�_�y�~�q�����������hnn.��---o����0�����W�z�-WW����O�4�����Z<c���z����{SRR���{���F��uuu{��=~�x�l���Ww������{��V�Z����t��E�u��Q�t�������q���'M�TYY�`����w5�p�����b��U�;w666n��]TTT]]�#Y������9����<�6�-?I���&�����b���*����#����N�211y��I6(����tss���idk:1�O����4	
����i��IEEE
�������?~��K�.^�hjj:j���T�!�%��mmm���sss�/	�{w�����$	 	e��MW\�-��m������
�j��X��
nX���h��"�V-�u��Z���fQD����
�*�3�?��<�*�q\�����2����s��3���L:
�DRK���l��eee����t���`������+iiid��%$���L33����T���enn���i0D����QO```dd$���utt

���<u����gTT�����/������T*[S���T>��k�.�T�o�>KK�u��=��)S�L�2�d,44���F,3����(�YCvW�=��(�c���'IR�Rq����l�54�h��


�0`�P(;v,�&w�>lO>>>3f��J�7n�8t�P����j���q�222��6����-J��]�n��������VK��I$�K�{������m������oC?��szzzqqq^^�SK�4��-��1���?�m���1�����W���fjs����Eu�m�,X�����h�i|���+V��mR�
����_{�5�H�h��I�&1_�=u�T��=W�^������mtQ�����2����>}����9;;����kE?��C������{���b�
�J��E36��y>����������h����������o

b�������:DDII�n����	�8x� I�>>>�|��S��y�&��[�<==�Z��u��>s��7�xc������X�X{�]���A\�~�?�s�������D"�nS$=~��$I�����1c�������@�dQQ��#t{�~���FCb����1<<<..�����s���7�|3""���c��!�7o�j��<~��������wqq���MHH�J��aLL����u�7n�ptt�>���ggg;v����nnnd�X�\���dii��{����O
Y_e��m[�vmRRRSS���{���7m�D��o��t����WK����w'''/_�������|}}}MM
A9992�L&�������l��ATUU�������������w�qqqIMM��INN�������s�\�[���7v������'O�<yr��AR�400�����J��!���c1���R�4
���^+ZUU����Z�J��	-�?})�r�H$���_9Njj�H$R(


4!��7n����0q�D����o���/���999k�����1�G�R���������DDD����W�y|��7���+W��6���g����_?>���g���b�y���G�9z�h�����}j��*���e��F���r}||���t?�<~�x�P8t����t�%�Q�c�������C�����@ �x�v����� �
��}�JJ����V�T��/.w��9;w��v�T���y�������\��l�������������+��������9��qrr8p����~����R����[�n�^��������I�����q<������S��R�:�K��P(������}6!M��B����������'I���o���nnn~�aXXXbb��TYYYW�^�?������g���K�,�=����$���_�����������)�J��!t��fffn���E���Ei����4�]��f��?�����7oR�|��������������{��#��c����P��=������T�������!!!��������_WW���trrr��]�����������EFF���_|A�|�����Cqq���A������������4�`W�u�u�277www�����?�R'/1K����&&&r����~��Q���
�����K�.4!v	�5���������c��),,7n������M�I���777�?^���S���u�}�������g�������*��55T*�S�N���8u���o���O��~}���JQ�T!!!������]
�~Q�������,,,6m����u�RI]���m���(���n�w�Y��	[����m�����w�III!"99y������I�JKK�����������������DFFR;�Zm�.]�r���9A������///����g�+�T���]kff��w�������uA�PH�J%���Z(R�����%d�������8qB����`��a\.�&�$sVV��Q��u��������������W����������;w��t�%%%b�����u
U*���o��y��9ooo�������7��$I���l��������M�fkkkkk�|�r�Tjkk{��)v54�Ei����4z��]XX��|����'O�j���_�|Y:z�(�'�[Q�c������=����[������?���xj��y�$��+Wrss?��S�������[�^��]�������n����^�����gYYYAA��+Wlmmw�������`W�5M���x������S�>�k4�� !x�I$�ZZ������%%%w��-..NNN.//7������������=p��������ZB���?����c������GSS�������;w����/^�X*�>|X(���L���w��=//���;?�������-[ZS��+W���VVV�����N����p�B]]��U�D"��)S��)
���+W:;;K�R�\�����(�\CvW�Fzz:��KOOW*�W�\����g�}F�v��aeeu���'O�9;;/[��5#�fl��$I666q���M6��l�������������O��w��j����a[P����w�������w�S|||>���J���?��3������������t���X�o����mmm���I�@"�^����-))���HII���(--eJOO���%^��,�~I�LII������E��������������g��<y������988���0Yjnn�����sssKJJje
����Z������k��9"�H$-[�l����&Mz�a/�[��mqQ���,�2�s������B�m�����������k��U�Q4c���g�l���zyy=�W2�V;x������a���,�~kkk=z����j�����kO�k��=G"���'�N"�p$���4����g�����~��'�X\UU�x��1c�<�ic�x����vA/@G����[[[O�4���������?�0aB�x���_��-77��^������a�����/A����/^����r�"�����U�j�
�a��{�������{����]�~�a����yyy�������/_�L�$�R�$&&z{{w�����k��mC�!�����7n����sK���s�[ll��T$I��������|GG�%K���r*�V����]]]-,,������U*Uk��]
�m���o������z�����h4��5K��?~���{������W��3��������O?��dL��T��{�Q������������s��������S���w������hnnvuu������n���^L:�����D��(��cw#j��>��S��<�����������,^�����I�'O�X[[�V	�(�DRK���l��eee����t���`���499��?��w�����SRR��9c�TKHZ���fffR�4++���<33�`�]����C�
0@(�;�d����������(�J�	��]kgg���#�J%���mxx8���z���/^���hJJJW�\i0aPP�����bWC���|>��]R�t��}������cRP�Rq����������:::���VVV�:u���3**�`6�R4��Ntt�H$:x��T*MLL455=v�C�R IDATjnn��������������

�����L�1��z1��g+�n��4S������M��SW��o��FDhhhQQ�o��������2)x��	#�*��$�H8�����fm8??���f����fiii]]�[o�E�t�������B���O�<y��7�K���'�(���y3��h�����?���>�.����H$���/��������2Y_�)�p��;w��������n����<~�xjs��>���#��w����{rr2


�������O\__?p���[�j��>� !!����u
	���m[bbbee������3����t�B��e�LLLV�^M6{��[�n�������B������ZYYI��
~�����Rj�J�2555X=�R4�������i�V�\Im��9����Z(�����f��K�.Q=��7���N�>=c����g'''WUU��7h�<��e���[yv�����z�nD4}�n�w�\��nDF�����k��������OKK377�)����&%%���y�&��:�q����\�/|����D�M�H���c�$�C����u_� ���t�44�X I���h���=���p��F��	�KHDUUUYYYHH�U������kjj��}__����Vk0����%+�Z-�H$���������srr���T*Uii���G'M����'N�P(��]�H$������jM
�o��t����WK����w'''/_��
����3��q����������555A����d2�LV^^N��=���oFDDxxx2d���LjHS��{�Q�T���b������c�����������s=�\.		IJJ���d8���yv����Z�<�I�n��4��������4����47"�z��Iqq�{����333��}	���e��;,,,d������*3���R�4
����SZUU����Z�J��	=��������o���	�5���Q.��D�_��������D"�B���@b�� ����4�T*UMMMpppnnnHHHDDD\\��aaa|>���qqqAAA��9s���5������
4h����g�f�M �Y�������7((H��(�����?^(:4==]�K��o�)..�=�J�����|� LMM�@ ��������kk��������k����Lh��s����c�&&&^�z� ����]�vQ��6n����0q�D&	iz����+::z����G�f>���yv�����+�����mi�����!�i��R����U*�������F��&?x��$I����&���������8d�Q�c��K�gcc�����s��m�&D��K��_�>|���]�2L��P(������}6!M�]B#����}�������7m��t�R&e###g��YPP�V�?��� bcc�����93h������>�����?:��~�����P�����R)��$YQQ�h�"�����g�Y�l���[��������>$I��������[yyybb��+ZY������u�������y<^���'N���+^
����i���3��z������L�K����^4e��/:��
Q�S������5�T�����oxxx�2d���:����L=�N��
������q����%f��_SSS�\noo?j�(GGG�BabbB=��R�*��j��={�����@;;;�	�5���������c��),,7n������M�]�v�^^^>d�4I�����^�xq\\\xx8������.,,l���|>���7,,l��MLN��%��!���YQ*�S�N���8u���o���Nsrr����n�Z�6�6]�R-u/;VVV���O�<�w���3g�\.�[�:�nnn���lk<��T*UHHHbbbK������z��Q+�b��-M�������A{V��FdD<���{�;e2Y+��V	/=��B������������
���j4!�V{���\���o?��R�R,p8??�'N�~����`��a\.� ���F7w����_��%%%b��z�gKH�|���?�����Hj����n�L������T*�5�v�������u�E
9�������u{�=ZYY����RU�8q����;������������$Ij]�z���E������^vrss���'O�L-Degg��&(AYYY�F����[�{����u������i��Q�r�\&�������o���F�
v������<��}��2�!���k�[
�i��������QB+o�s�����d�3t��={����S{���/��g�R?v@����?���]G�K��j���GUUUUUUss���7o���{�$M����������G�������/SSSSRR�R�������/^l0�.�L&kjjjjjR�����lp%�������9r�����9!!a��%��8���S��_�g��Tz������	&P�'S�NMHH(..�h4�/_����N�j��$I��{w���j����K?���n-�E
	�KKK;t�Psssqq�������C���[w�����w�������������f,\����[K�,�s��D"����?>�R�����;w���WPP�����7���:,,L���g�2�(S����5d����o��#<<����������uo��yv������<�I�n��L=v7"�>d7�;���<������YSXX��'�����7��������pppx��INN����\�r���6�U��O"��RRR���������QZZ�$��O?���RRR�$|.�����~��YXX���?--�a�EB//�g������hJ577������������%%%1��R����qqq��xb�844����
577/]��������W�^O�<1�p�����O���/���D"����
�.!��$�q�F777���W�U�V��jj����:L666q���gC'O������x111�s�k�M��eoo/.\��T���'&&�PB���?�A���b���`c7�Y_/&�T��M"vC�~�����7XL���+����-9r���_ t��}��E���hf��meeejj*����V�*��$�H8�������O[[[c���rss����K���~:=,�tzX������~.^�hmm��rE"Qcc#���������p*++_����/���a��{�������{����]�~�a���B"�����������F�aR����Doo��]�zyym��M?�m���}�������+**J�������5K��?~���{����i���W��cnnngg�����d���CvMn��A�d�~���7�(Mf�����jutt���������{tt�J�j}��;�XL"�$ccc����|�����%K�ry+geK�H�V���zzz
OO��;w2I��o���C/Av���?��������]��zY[[o��A��e��@�F����D"��UVV�c�����{��]�t��l0t����~����+����������N�8a�TKHZ���fffR�4++���<33�
������]�vI��}��YZZ�[��4�&!M��J��r�����1���:::���VVV�:u���3**�5	����L���p������c������;WTT����~��E�FSRR����r���$4��b7���]kgg���#�J%���mxxxk�L���h�Ht��A�T���hjjz��1���/_�����G�Tu�>��4~�����]�n���_�^���������\��H$�D���O�6���occ3p�@j�����������]�~�������
�?^�P�9��TK���i����y3��h�����?��� �e������^��
��=���[���;�r8MB�
�@�P(������^s�q�����oUU�H$


]�d	���O�a��_~������T�T����g�B���_������G={�\�r����
������	���D"�_|�����<x��==!�s���;��wONN�BAAA���999��4#��$��u���������3f<|�0//�uB�I���1m���+WR�g�������o��'\�p��;w����B�N��DEEE�>}t�������K�wT��������u�V�������X_/kk����/Y�����e���o��^����_�\WW'�t�"�����$I��\]]�u_�$�����������`BH�,**1b�n����/\�@��2&&F�/�A��q����uB�s�������!"''G&��d���r*�m���k�&%%555���7>>~��M[}���7�|3""���c��!�7o�j�T��o�����������BCCg��q���vnr{&$������,$$�X��EB���<yrNNNQQ�J�*--=z���I�Z����Q,&A.��,�j�D"�H$��OoMB�I�R��E/�X,.,,4�����555���������oBB���4�N�����'u���^���'N(�k��I$����GEE���j�s@ga`�W�Ri4>�O�W������k�Z�JE��x���={��=��_?�XL��]�r�H$���_9Njj�H$R(


O��7��
d���������|>A����@ �x<*�e���9���������1�����o���/���999k������B������|>��9*���������		��u���
����5g��Y�f����x�A���9s����I�b��(�H',,���O�<9...((�5	i&���c�^�JDqq��]��Eb�7������������������8��N����T*kkk�Je�C��+�v�22n���5k���}}}�������s���>���0<������P?4H���iB���b�����/^$I�W�^LJ�#
}||lmm�MH���e��n��o�>ww��'�	��k���Y�f��5�=���Z����[�'I���o���A������'&&�X�� ���������|�LF%�^�?ir{&4��1���+666==���3�
*++����x<��C{/����Q��D���3g�,((S���|�I+>��u�������y<^���'N����d�TVV��s�>}����i�&�hB/k$�J����x�
����\�V�~�����P�����R�S#��������/1K����&&&r�������� ���o���t������������611���?{��I��],--���<x0}��1c��s�N�@`eeE�T*g��YTTt��)oo��$�p8��b���b������/T����{���MjuJ�����|��J�����g����I���������������[�|���C	����
[�t)��/}B�~D��D�$������lff>w�\j5�]��XYY���+�J�L��[����=z�h//����@/��K��`�/���n�@����211��\.��o�s����o�
�B���VWW�B��B���C�***��i�\�����b�������8qB����`��a���J��8q���7��;��_�i������{��.���<yb��������u]z��U�Z}���y������ �����I��3a[��?�F����c�Je�4	�
��n�$������dSSS]3Y$�����������u�F�dvv��Q���;w��t�%%%b��Z��	��}h�������gR����\�V�k���>_�~]����s���kUUU�����K�oX�^b�����8s��P(���{����[��
f0����_�v�������������{����.M)v����?�p�������rss������C���[w����g���������p�u��:!M������9s��q&L�~���)S�x�����R.��i��%K���.]���W_��K�������q�����X+++&�{j�&�gB�LF-���j�F���D�����m�������N������[o
0�����~�:u*���@K#��$�p8S�N]�~�����!C�]�3a�jU����q���/������w��+V����cRp�����o�����{BB���~�$�R�!�I�w����y��***~���E����h���6l=z���wQQ��]������\���H$�����ddd���ddd���2	=z����3;w�LNN��sg~~����$|.�����~��YXX���?--M�������ZXX����>D���� �#G���K,GDD(
&	O�<������bbb�j5�?''������a����/_���HHHh�&�[B//�g���?!�s577/]��������W�^O�<iMB����n)�����'�CCC���[9+i&QTT����@ �p��l������=z���xnnnIIILB��;�o���/��0y�����S� ���������^r�<<<���_�z�Z�r�R�l�s�K@"�p$���_�����"=������o�����w�����������C&�]�r�{��<X�|����}���.�����8m�4gg��#G
����,�	����_h'vvv�����~:=,�tz/���������\�H$jll���WV������:u���S1J� �\��Gbb":Z�����7���������w��u�!�F�9|�pNN��ba��YZ��������+..��� B�V���zzz
OO��;w2I��o���O��������G��IB� ����v�����m�6��N�p��m}��577���WTT�F����d�~���7���1z
���E�I����uww������K�,������W��3��������O?��d���	�^Cv�������{�f>a���m����l��y�
6����>%%%����m�����������������}��Uj��5	�L&��d����?h-�DRK���l��eee����t���`H'???--m���L��4D�Rq�������GGG�D��J����DSS�c����|�r�F=J���cBCCmll�b1���fff���eddH����,ss���L��N�055������K*�������r��uT������C
�c��%�1z
���]���]kgg���#�J%���mxx���@������144������S���QQQL��"��k�Nss������d>a[S�)S�L�2���l\


7o��y���C����?�Y__Oj��\�|� ����6m�{����o����7ndgg�D"��i%{{�m���lI$�K�{������_������(����v�:}������R/������Z�vuu�B�������#g����������4�:u�g���W�f�Rk��#F�����u����C�:��_�b�
�fppp`` �������{���AAA�WO�^Cv�bW�-[����W�9}����G�4��_?`����J'���%4z
I�������onn��G�+V�T��@)))�z�jnnf8a_��
�b�������H$Z�h��I��,��79>>�O�>fff����~��V��_}����q���
]ee%A�.]j�S�;wN����h��w�}��gO33���{��������.��?^��E�8q������?wrr���^�b�X,�6mu�������9h� ''�)S�455�w^�������D"�nS$=~��$I�A����p���]�ta��������� rrr�7g���S!�Jeff�;R,L���������������7!!A����r�<$$$))����a
���$����#F��>����4$ IDAT�ZBK�N�� ������W�6o�����H}���*++		a>f���,����A,\�p���A��j�D"�H�O�np:�8z���o����1d����7��(���%4z
�o��t����WK����w'''/_��yg666����������E
�������;v��]77��2�M�m���]�6))���i�������6mz��`A�I��k�S���S&������TTT899�������III


?��cLL��u�XW������j���;w���������c�v��u��-� ��]�<==����������'$$�'�K�*�J����|�E�UUU|>_���T*�U���������/4X�>�O������@ ��;vlbb���W	�(..��k��d��555������!!!qqq�htt���#G�����M���(��E"������pRSSE"�B�hhh�	u��O���o���W�\Im�X�i�2<������0>�?y������� ���������kk��������k����<��F�a|||pp�����B���C�����
��q������NX5LOO�1cF�~��|�g�}&��T�&��-[,X0r�H.����f�_���T*kkk�Je;$d�Y�eW�gKq�\�@@}�h4K�.������o�����9s��=�K�.s��MNNnM����w�}���M�P��������{� ���g���`� D"���#�g��U���q<������S��r8�Pmm��;w����HhD�����������x����8qbRR��RYYY��}������M��.]JDaaafffii�U��	)B���������>�	u��$I.[�l������swwo�1z
��������3g������O>��u*�$����1���V^^����b�
��2��X5$I���b��E�#��U(����M��/��e�d�T����;����Ix���5k��Y�Fw����V��r�m�U!�J����x�
��N�.D-�>�:q���)�P(����������={R�f���_���n��A�V?�J�bkk���#� ���8����RSU����3��'x9�wgSSS�\���B=�{��m�.]�p8��BZ��z����)��m�<++���t�R)���u���G�M������C��������D++�����	---���<x0}��1c��s�N�@@-���	)J�r���EEE�N����n�%h����Z��$������lff>w�\�+NNN��w�mR������������\.����~c��������7Z�����6�$I��3�L4	-,,V�^��0�@`oo�{*�M%%%]�|���\�-�-�C�$��;�@��s�������H�)t|�W �B��3C���B������B���������������/_�,�����O=�D���rss322x<^�n�H����5j��Rs��=p��n���D,����?���l��i��������/�J�����N�j���������=��
�r�4�N�� �J5q���7o�;w����mTC�b��$Iggg�W�����)b�������uWW�^uqq��,�����F�!��qww�|�����G�&&&2L���5j��n��1�������|��M��c��?M���{��l���<y��_��������O�.������okkkcU������SSS��}i�!�������rii�����o=333�\���}�6���^|���������jnn�y���[�t��R���f��������!F�mccC������3o��������{��y���aaaL
��?���#w�������,YB��!Cn��]����p''���R__�vN���_������H��������-^��`�S'\�n���gw��mbbRWWWWWW__O�d2YSSSSS�Z��h4�g���F�!�s��<���:u�������#�J�;3a�����w����n�Z�d��;w$I||����[s����A������:t������x���w��a�����C�a8aY�p��	���Kaaa}}��5k���Z�������gdd�T����1c�DFF��_f�����a����\���uuuO���������'N� I����?���_|A��� B�VS������{Q2���S\\��e$I�!%%%)))���C:�����s����G���H��#G�
EEEQo
�p��@sssxxx�=x<���[RR�s��������$I������������iiiC�7����SC����
yyy=;�8��Mfq.v�W*�111...<O,������3�4N�<������bbb�j5�s�Hh��$�q�F777���W�U�V��������Dv�y
������#�D"��e����;i��V6Y7��bqDD�B� �G��Y�{w�	��3g��\c������������V�]�n������e�>}�n���j��s��KJJ�i2a�oo��>���/�x��wu-U�T���'����"I���$??�$���������4v�X^��#�H���;���s��	A���r���~:=,�tzX��������K�/^����r�"������n��Fd����b��{�������{����]�~�I������oL�@�dll���;��wtt\�d�\.�?`��YZ��������+..���0�311����k��^^^��mcB�W<!z�)2��_�~����������o�{��hM�477�������y�i�6,nDm��E566~��Wnnn�������V�R������$I-����;v�����w���K�g���'OfggW�y����R-!i�]����.''G*�J$[[���p]T�Rq����l����L33����T���enn���i0���xB������C�
0@(�;V?�����'00022�C5Y'44���F,3��4w7��H��{����o����7ndgg�D"&���D"�H$�����|��R����uuuo��}���D&�
>�������&�u�Vgg����S�3f�x��a^^A�@�P����ZYYI�0x����7S��-*..���?�CH��'Do�srr�D_|���QSSs��������f��K�.���u�&SN�>=c����g'''WUU��4wv7"�'d����nnn��������,[����C�����)�%77������D"�nS$=~��$I��R�T(�N����9v�Xee%���;.��}�j�D"�H$��O�B���555A����d2�LV^^N��$����#F��>�����&���xB��S������BBBh�ill���3����M�6�ryHHHRR���%�Ms�aq#j��,���S&���}	����prr��'@d`�W�Ri4>�O�������k�Z�JE"B��*��=z�����������	[����0>�?y������� j'�����A���
�@�����466��r�H����r8���T�H�P(hBH��'Do<��D#b���'N�P}H=r����G3��4w7��H�����g�F�t�����o��F��RY[[�T*�-��C4:u��5:;.��x<���
�����p8��a�������������������S/#�I�Ndd��V�\������
�>>>������&���xB�C
������T���������
6�B�z�;����EEE�I����������ht�v�k2tv�~MMMMLL�r�����Q�
���I�.]hB��������Z���J1G��������/^��j�����433{����1c
�����C�@`eeEB�W<!z�E>|���Y���U*UHHHbbb���2IJJ�|�������gO�@�d��_�PH��$���Z(R��������������YXXp�\�R,�$�����cjj��gY	��p8~~~'N���)((6l���	!�+�������F���[�������/++�6m�����������R�����S�^��������>���]]]?��c�����������~�������3B��������n�6l��A������VVV>������?�R/���L�:u���C��v�ZLL��	Z�4���_~���C������������g��!$|��7��d2�/0�j�F�ijj"�������:������O��}8d����o�KNN��}��3g^{��W����������H$�ZCJJJ222RRR222JKK��jjjN�8�k�������w1L�\$-�R�������bqhhh}}�.���H��#G������_?�������1!�+��������������|~bbb��C���x�X�0!�������	Y;v����o:*�D��H$���|}����������E/tvX�������uA@]]:�	L����������?���oD��^�)�zlCx���{y�~/^�hmm��rE"Qcc#�P{&x���O��~�Vb��[PP�p����������������_�`AXX��?��T*��bm��YZ��������+..���`j��������]�v�����m�P{&��d�A������B����2l�����:t��C�:!�s�������C���N`�0�.!A�.����'�6J�_�@_J.�>|��)��$�q�F{{{OO�+V����$��mllD"�����3.\��/iv^��L���������9s������q#99�������>����~�z__��3g644���>|x�����XS���.]���0`�P�%���Z�xqZZ��a���;7k���^{��?��gB�U�V���YZZ�P�Y��������m����WXX�����D"j��:uBv�b�_�%o�Xq�������k�u����^{Db���g2����#_�+�~���M~YM����3g��^�����~������������?������+�|�	��]�j�����~y���;v�����w�=z4���{����H$���4G�������>�����{����#""�C���g������B�����`��X[[�D�B�_+R���������L0x����7S��-*..���?�C�������p���y��M�>������?���	=�;��3`����8j322������u�����`�����~��m��]���;��o�4����L��x�$������{���:��KA��h��:��O�L������V�f]B�������0a����B������9������t&�f���NNN�����y���������&!�������4h���O�	�r���s�:H
���(E?S����Dnn��>�$y��-777�77��7oj�Z�A���YYY��-[�v��c�H��O�����555A����d2�LV^^n0�n	I�,**1b�n����/\���hhB���"��?�������]�2�T���R�?)���+))�j�R�S'dw.j����
����0++��������O�!����wt{.�k�r��r84!v		�������.�-{h�&�������y����3g��������u_�Z}���c��M�4�
�K��|��G��]�{�.���{E�<���_�\�R�,,,��?����:u���B�V��r�A�?>{������E��}���:t�>!�6��|>�O����@ <�`��666��r�H����r8���T�H�P(hB���;|�����
=WSS�B�
�{�������k�P(T(���4�N����t�j��e��u4h�G}t��A���<^n�\����%_{���:��g���7�	�KH�JC��/��/�����������>88x�����%�&!;J����V�'��.����L����u{�\�R__����B4hJu�����[���P���n���t\��YXX�������sC$I����?������������

&|Y	�B[[[�h����4��IXRR�w��5k��P� kk���D�g�A��	Y����K�:��={2��A����I�L���'����m�K<�>l������=22����NII�_�������;�����~�@T��)0,
�� ��U��j]*�EP�Z+��[�V�����^�U�/7��j�D��� `-�l� �,�JH������e��a����1�{��<�9gf�|r&����JAA�!(���e�.s�����o�>��D�H��D���jX}�������>���S;.�`��w�~��wm��m��yE�q�J�*W�EQ�R�j��W��������a������
�HU�T�\����+����/�0����>33�j���Xlo�C������;�x��'wO����8��c233W�^��K��;FQ4v�����*U��b����n���+���	���������%�GU��/�L-XE�����}G���;��ho���C>R/�2�rj��x��5k��y���V�\�����+T��/���d��(�rss�U@y�!�b��
��?�t��_~��������(:��.\��m��|�MVVV�
�ouD��bg�y����_�d������K���J�����3�����w��������~���yyy��MKRJ>�3�8���>*]����y�����t����x�h���RAAA�Kn��G�?��������������%�d���C>R/�~����f��
2�tI��S����///�W�^M�49�R�=�����^�F��%?�����[��k��WN��D�����9�S���z�J|��G��e�q��6,//���N�3g�G}t����Y:����}���c�^p�_���	.���2�:R����s��m�����S~~����_}��2K������������s�=�����&L�����b{+%�������ol��������{������=�\������x<�|��#F���s��/���O��2>7c�?j���M���o���������/�,��0�b��X,���
��
��*�EQ�mG���
�m��Hmw��Q\\�y��(�233w��{g�X������������e��������Y����7n\�d���K�y���S����k����|mP�������i�|�)S�L�0a���5j������o_����_~�������>�c���\rI�
pI���j��%_a��MU�T�0aB�N��_:�
�}��'�|r���yyy�������S:4
��_���a��
6�����k&)���^x���


6l��������+O)��)����o�X�b�Z�^~�����+����GIOOO���Q_DQty��ZW���e������o���������V�n:5/��]V���y�W�P������7o�.+���K]tQ�nEEEC�}��gV�ZU�V�������� ����^{m��F���?���f��E�1�s�g�s�=m��=|F�x���I=O:��<"�
�#??�\��������K��a.����;��G�K���BD��m�_V~~~�:�/@�D�����)�Z�j&! ���3	x!���]���}s�u�]w�u�+��~/^��;����������E��Sz��w��_����i�?�l������A��%�x��Gm��qFFF�:u~��_l���<��~��'s�1��7w-s
 IDAT6lX9KGj�4(�=����yR�v���+���!|j�[�a��D
6�Y�fGu�	'������V�qm���E��]vY�g��>���T��yj��s�i�U�\977������e�����i��w���A����5j���k����R�V�v����'O�������O�J�po���?�)�'�������}�zsrrv�WAA��i��������-����Z�n����v��O>�$##�n���K��~</m2e�����2��{��w��uU�V-]����������>��E�9s�t���B�
�?�x�>�������F���]�i���p�
5j����s����05k��m������[�$##cNJ���d_�5<x��S���c�&j������{��g:t�0s��n��U�\��{�M�bKr\7n���~��_-_��~������$��SyF���_��k����������K��q����c��=��U��z��������M�~��7�xc��=_y��D�k��?�p��*��Z����f��oL�s�����K�x��-ZDQ����|G�]��f�*�]����.]�T�^=77��4����~,X��a�D4{���7l�p�����IJiii�


�l�r��g��p|��Go��f�~�F�U��Z�jO?��%�\EQ��u/�����g��j����z����Kb�����!�X%I�Hm�}���}��7����[�n%%%��U��][�V�c�9f�Rj'%�YN���FQ4l�����/\��f����w��*V�x0.�>�����a�m��f����������$1����z��	��7�K4�q5m�4;;�o��|���5k�9{�I�d������������~�m��
p�5���f�����o�6d��(�Z�lY�f�A�m������:��U���n�����:+��:�9���Gi�J�*����5��lT�R�J�*Q%�E�S�N9����[�^��?���|���o����7�+���������EQ�V���IJ�v�������h��4.�V�j��m={�1bD���R��~�W\����������'o��?���D����}����')�
�(4h�+����{�}����5z����sF��]�f���/�<77�U�VC�-))I��$?�I��Z��#G�{�����o
_|��Q�F�����L;d�Omz�co�J�����o~S�p���u��)�bK"�q|���={�,��8�O��s��#����+'N\�~}�>}�u��d��C?�l��y���?���v����cq�EQ�f���S���;v����!����������?��c/��R��5�@@�oQQQqqqFFFaa��q�


222JJJ�����v��`�������[��|�C�\p�����������k{��o�1y��7n��-;;���^��b�>�lvv�����w�%)�
�(z����u���E����������J����5k��t�M���={���/��c��|R���2���

t�M7]q�YYYm��}���w���%z����&F�}���G�9s���������JOO��!�'Q�9����g����I�����o����h��Y�~�)X�re<O��{�}��u��m������~������N:��O����?�����#G������������<�����z�LTaaa�n�~���v����[p+�w����W�^=###q{n,+O���d�����v�>5�W3f�;vl�o����{�w�>y��~�������[o-O����3�8#''g�&)y
5jT�N�����j��q��>���V�X1d������vR���2��O
����y������%��U~0.�;���7�0��W�x����{��������r��v���\>4O�$s�i��_����&M��uk�[y�m�v�G����+'�c�u
'M�t�9�4i����4h0i��%K�L�8����:��s{��5|��D�s����-H������c�������***���k��m{��w��I�VF�[�R����m�������U+���K����U�X1���T��7�|S\\|�q���ajPTT��g����W�Zu�+���Z�j��U�e���+W����{����o"V�R�r��+W�����/���(��������U��b�����0���	���dS�s���7_�jUQQQ�J�R8)e��$�J�azz�Lw��S��$������������w�����N�2���O.�������{%?�W^y����'O���a���������G�������/_����[�&B�C�~��=z����������={�����O���Z�j�({lxX��/~��������=�?�*;5���Z�fM����Wgee%>kNRJX�lY��5w	���j�|��'��y��]srrrrr����������)S�����u������*U*��a�b�3�<����/]2y��v��U�P!I�HmEQ��u������x<^��?{�����o�>�5kV���+U���IIr���+���X�q��s��)]��w�-����^�|��Mo�a$|EEEW_}�W_}5m�������moR>�C�$J2�;v�����o����
FQT�g��aj��m�W_}�t��j��5w���%yyy�z��������g��{��+W�,]��U�(��w\��lp{���c�>|�����W/�V�����I�&S�N����Y����+�,Y��]�2K	k�����-��n�z����G�5r���S���Q#�]w�uO<�D�&MZ�n=���~����*3 ��������m��S������G�~���t�6������?��'?�I�����i��r���n�-33�Y�f'N:t���?EQj'%�YN�������o��]t�y��7o��=z������O����0�����w�����?���������'N�����%����n����w��Q\\��D+W����v(�D{���+����yEK�.}��G�V�����������������������o����O���5kv(��7n��W_u����Gm����e�~��_5j���SOM��q������7�S�NFFF��_�ufffi)77��g����H��]�`A���5j�<����'�b��?~�����5k��1�<���1cf��]���u�F������S�=����A����_�p���?�pnnnzzz�������a����y��gZ�hq��G�r�)�G�.g��l�e���o�9;;;;;�������w�.]���e������A�����51b�������$�J�a<<xp�F����O8���zh����`>��-�0v����Y��./5G}��\�I��y�����������I�d���'//�v��:t�3gN�~��>���C���yj&L���M������?��;���ys��[�xq�n��;���+���t������J��8���/�>� �O�4i�������+I�2gc��YQM�>��������O�����G��/l��������i��0���U�&������
f t�_���~�'��� x�_���+�]�x�;���������;�-*O)���;����~�����?w������4L����O>��c�9�y����
+g�ph8h������������l������A��%;v�x��G�6m�����i����U>�������oZ`�U,s�e��}��g�[����^�v�'�|���Q�n���/����������Z���
>���(��5k�|���7����=zt�v��M�v�
7��Q�s���K�I��k��i���w�-]9##���~����[��j���K��?�i�����z�_���=z��S�c���u�p��o��n�i����^z�o���E�j���������i�&��&M�^��i���x8{������{���K|�A���[�j�(M�>����?����o�7�����>�e���������3g~���K�I��o�}��e��_R8y}�Q�n�z��1j�������&M�t��u������w��i�k���Z�g�>����O�^�6��������y��������������z��}��7���+))Y�~��U�J��M�U
�����~z�Yg�.i�������������Q�]�v��5�_~ynnn�V����7c'�m���={�1�J�*;//**�\�r�������1��G�2��������������q��ddd���%)EQ��A�������{������������{���J����m������K/�b�g�}6;;�������������	Y�f�M7������g�_����=�Xy������\���K/�t���_~�eE3g�|����Y�B�������[�}��U���\/==�z����sc�X���y�
;v�X�Z�
6L�6�B�
��7/O��dee�q�999�7LR���7�t��N:i��C����{���1c���cg���{�w������O>�����SN9����1b������&M:��s�4ir@J�W�]��*UJKK��m[�Z��?��:u�|���iii+VLR��h���7����P�B���7n�p���
S;�*U�T�\y���^x��3.���U�VeffV�Z5I�0i�����7_�jU�{����z��9|��=v�Z���/��y�����O�:�B�
����#X��7++k�o^�zuVVV���$�x<��������&�*�X��3�|���K�L�<�]�v*THR:LFQ��w�7�|��4k�����W�T)I�O>����?���kNNNNN����_XX���3e��(���������~������W^y����w��#//�W�^{�y7�p��}�m�&M�N����U�f��+W.Y��]�ve�����h���5k{��7n\�xq�z���*5������s��m;u����?z��W_}���a�0���n�-33�Y�f'N:t���?��[����.]Z�p��Q#G��:uj�5�(Z�lY���k��y��'����^�zu�~�\��ab������_�|yE+V�(((�V�Z�j����������i�|����l������n��I����,����-��ukfff��u�5k���Vf�=*3x��g�|������������n(O�ph�u��|p�����/�������o���}:��<xpAAA��|p��6lh����A�Z�j�Bx���v^r�}�
8�������/W���sC�����W0��O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/@�D���O�<�/n�
s IDAT@�D���O�<�/@�D����bE��M3���[��,�>O�<�/@�*�s�M�6����y��|�fCE�s�=1����u<�����vN��g�}�}q|[q������h{I|������n9�i,����	��sV������)�W���w��9}Z�]F��]|�O/�|��'wZ��b���N�5���w����U�f{lv�����S����cWW:��m[����������L������3f������G����i��<��m/���6��W����:��Q���{��g����Q������B,:&�����U+�������vT��;���w&�����9��K/���N�����[�
��O���������/[�����o���.���>}�<xo�/_^�v������{��w�~oo��6-��zXy��'�����{��I�&�v�i
4�5kV��-g����z��_���?����7�9 ��f��}���^�����ohvt���=��/{t�>���m[���7o�|���`����Q��@���7�n�z���u�'N�����T������=z�8��n���1c�t��9�����-���~B��2��M�6�x��S�]��S+�B��V!�B�E�X�B,��]�Q��#^xs��d�a�2+^���ON�����K���}�)Y?;5�����2)��-[��\�M����i�;v���'/�����������Z�n]PPp�m��my�.]�?�MM�j�y����O9��2�u?�P���M��~��[n����z���,((8�����������y�^x�����(�~���
2d��mI6����v�W��_�Q�F��v�����gK�.�c�
�j�o��}������C��'l�5������~{������,�@G�y�9�w�q-Z�x��'��xE��m����7n\�N�s�9g��	���a�������4j�����~���K����/��,Yr��77m��^�zW\q�_|�X>{��N�:������v��y����o���]+W������4iR�z�������O<��q�z���u�]EEE��,����j����]��>�h�c���=��9��s�e��\�2���7����;v�B��(ig��_��V�Z'�|r������{�����7�������;��G���u�
v�WT�R�������-[��]�����+V�s�;v�XXX8s�L���a�����

���?z���-[���k��{������%:9��~_~kb�N��]+E%���8^��QO�T��f�������&&iR�R���3�=>3��(�]P7�'�2k~��E�������s:�w���b��+��f��/��b���^{m�O�;v�g�e��5m�4��:�z��?�Qu�Q=z�(��i��x����6|��W�:���U��)���^s�55k������9��S�x��$�v��u��)o����E����o�������z��	&,^�x�����|��7�={��#�~����>������}�~��s�����o�����][��3g���������;��^�z�1"�<��_V�Zu�)����[3f�������k���U����9s����O:��[o�u�mw��+�]t�E�F��e�W_}u��������i��M���?�!���������n�z���{������
����_~9y����'_~��C���h��-O<����[y>J�E�z�V�^�x��1c�T�R�c���~��.���s�C������+�����=������wY���d����w����e�n��������/��]�v����9�Qu����g��j	?o$y�^�l�	'�P�b�(�5jt������������V��~�-Z�]�~I<*.��(���(���K�QI<*�����j��y��$�S���?�����v���/������7�K�.m�l���3�)��J�*%������q��?��q�w�������?���K.��A�����4i��
�5k6x��N�:�j���?�i�����O��
�7o�^�z���n�����G?�S���W^�f��;���n��������/K{>�i����5>|x��q���m��~��W^yeAAA���A<xp���w�e���z���N���������p�w�������oO9���w����o!�}�;��q��7����������[�l���g^^��k������>~H�*h�A@�� ���dW���UQ����"n�z+.x���(��VP,zUE\A6�MC�!������MB��mo����?��9sf�339�=�y������;��%V�����p8���pX����`0D"�sx��uo��
		���IOO�u��;gaa�d2�9�����������/@$))������p��I����.�t��u����3���������������d���999�����E�|����fEE����g��y����
__�%K���������^uww����1<��e���v��ip�eLL��l�NNN���������@ $&&Z[[3���+W~��Q�"l���(00p�8����	&�?��������������M��ihh��������
<a��I������644t������L&����(R�����F�Z�j��_|�~���^OO��������|��~����@ �@�=���$IYY����������rti=���gc��)((���/^���dZ[[C�SVV���W:dccHNN���066vpp(--TVV&''K������L�B��'N����
����'���iii1�L�@��;*�L�P(�G�^�bEBB���;�|>Q~������4115j���������$GGG���2�f�211133����r���G�������q���ke��a��={������2������,��������0�L]]����������ooo�������`0����/�(v���E����H����c�X#G�\�f
L�����7o�l��QCC#<<�N����*R�������D"�X���Fx����1�XwP)Ieeeuuu++�xzz�����]��V���URR��i�&L�E'0�����L$eee�_�^[[���100P:�eEE�P(d2�
����������6l��-[����������7���@���
�hpsssrr��g��MLL�~�:t6>�����x��������;�,������k������666���������www�u���L��gg�/��g�0-���~�X ���b��:����a#�?�9�H,"���b1��___w��
������^��������ohh`00�pgg���_DDD]]���,Y�f��+>z�����<����=^����k�������~���'���O�6m���������)++�����o��u�����7��������1c���p^ww7��+..>q���+W�=z433�������p|y���i�����������Fq�l����[YYy��E�����UTT���<xPEE�X���^QQQ���	�999A<$99y������/++

�fy����g����w���D���w���q�Frr�t&�YPP�������{���k��{��9��
		���{������cbb�={6����***����a-Zt���|��Wpn�����BaUUUVV���2�X,

�5kVCC���+��2y��������|�	IJJ*))�u�VYY����c�S���3�/_���!}����M�6:t���1;;[WWW,���$SS���D���2������\JKK���d~t���7n<y����
Z���O_�|����N����S(���2�,�|����@ �@�=P<#���

F�=�#�'�u���BBBN�8������-//�q�Fzzz^^^``��9sbbb���:����s��666^�|y������Z��S"�=~�������5���u�����BQQ���&oo���	#!!���������:<<����VRR*((���;z��D&�aqqq������/^���p``��;w`D�H$�����&�GFFFAA��G��P���U����x���sssCC�
6���J���dR]]mmm
�������{{{���G���������V����YYYY������kjjJJJ,X Qw ))11���V���sPuW�Z+���Gyyy����p8�;e����ZZZ�\.�8��#v�"������������V��mmm������@ �������������2�aaaqqq3g���M����|��1%%���K���.E	�F;}�tSS���gw����b�2]'�<��4hQ��1����B&a0X(�D&v`b1��P�����

o���������l��;����B�����x�p��[[[???2����4y�d���"##UTTTTT���aE��X[[�����3GGG���OUU5((@���7o�P(���!!!����w�EFF�;�L&������khh��W�ZPRR"�����t�R333*��c����|��)�"�QHNNN\\�����X�x1�������^[[���dkkK�P�U���KJJD"Qqqqtt��G�0+**t��a��m�R�c����������e��`�u�~~~t:�L&���:���*TTT���l��AUU���nBggg�h=x� 66w]�?�>�h4�_|j������9��z������S�T<���1������W�\I&�]]]� �O���3[�n���T*u��
��=����������eee77�)S�H���sQQQ���$��`�5����2-&E����{����/�b�
mmmMM��k��g*888;;[,.]�4o�<�w8�@ �@ >)��
���G�%�:������ys��������?���/^���}��!C"##%����r__���/������`��92���wcF�`�>}���[��o�<y���x���t��4t:]YYY"q���JCCCkk�����q���0�t��s����!133�4i������nTT�����quu��s��������@&������`0jjj������w�^9N����o��f��=$	d
������p� 	�K&���p�YL
���K�	�K'T�Y���d��%^^^���+V�X�~��={.^�8k������HJ:~�xSS��]�������wAw���FDD��������(�C�����e����q��G�E����M�R�d�����s�7���
EII�W8) 2=&#F�����I��P������*���y{{?|����?u�,������;�U�$)))� �0���H��D�����=L�a0�X���������K��\�2:x^���Y]}�����q�0�0@]]~/677?y�d�����t�����m7hf|��J%n�
O�add�����%%%��K�C��544�YrD�l����"�By����a�>� 2e0�5zzz�����&&�tJhh(������?�������FFF�����?/..�A�555EEEaaa�O�x�xQWW�_D{�dee?~��f�����6�k���M�l�����`0�?pvv^�n�����c@@����[[[����/9<|�p��JGG����w�z{{�\.���7��	O7)������:;;����A0b�t6�m``�{V�'�����-'�brP���r����������.\�E������ekk���y��]cc����/��R�G@��@ �@ ���OJ��q�F??�������LFZO������=����=.
���������baa����s��W�^M�>}��]������r�����\��NhiiEGGO�8���Zbrttttt�����p�B���&q��]�������W�^�P(��
�J'>>����@��[pp���G���/]�C~2������������|�r|����N����W�������VWW'���d�������a�>uuu�N�m�W��������L&�����d�����S�N<x����������;*�
'��H9Z+.�A�nK�C���]����o���ni<�k)/�#��"���x"��S5g����
��[G>yL��P�w(�R|����]�jkk1�������?<�z���h�*���:�$�I���\��Y�a"���������	�
�PQU���Z�,����5�JYYYI�$��k600�4iRZZ��VTSS�g��|,����;;;�����E__�IPPPdd$�d}}��lD���^ss3>j����8�z����}�hhh���������)��M�6m���������c�J������]����kdd���������fkk+�����"466�Y�&//o��1��p{�����l�@_MMM��c������8q��������S����pG�LX,����'M������7o������i4Zkk+����Qic�����l�H_C---��PA�T�����s�X,�@71=����y$B���'=��^SS�L�=
;y�$��/���V���38eff2///*�*}2��;�@ �@ ����}q���D��@:���IMM��q���{�*�4������%�
~��}tttrr��}���ajj���'����c����_����?����!�����=z���w�����|>��y�FBr��m���zQQ�J������������������)�2����}��������=o�<bc	�\nGG���WB�����EEE���������H]]���������:�����N��HE�������������[XX2���1))	~���$�+q'Qw���666��nb�J����

i4ZEE\hZ@����[XX���O�	ttt�jm:�N�s#�9��x]�������;q).Z�(...((�L&�����D7
��I���2����'N�����o����S��T�o+������0��@,����eW��=E�q0LD��N����D\�{{�!*�T
�JQ a���������999����?x�����������Z�����FI$���;��������?�����o����������\�zU,�'O������x[�l�>}���~5h����p�����=w�~��0---��W�����'O��0a������2]�:::�����+{zz�T*\�������N~y�e``�w����������l|�ogg���_������
^f��i��������pu���8����IJJ��x���'O�����������D����GfD���e�'��������f��r��;l��+W���)qY|Y���������R�����������5\�����9sF��utt�������Oj�v�Z�)0O�>�7o���	y$HII��������}��3�;w����/^��g{1b�@ ��7�|���@ ���I	�a���_[[����YYY�XZZ2����x.�+�_�zW�#�E�^�*))
�4�F��������������w�<x0++KKK���*/^��z�j[[����o_CC������(��|~CC��c��m��i�&�5��^����������C"���u?>�_YYy��\��tww�=�J�����R�B����DDD������L����������[������\����9s���1c���}}}}}}�~��������������XSSS< ����k2|>�>s���W�8�?������������������pf��x��m||�P(�"��;V�X1t�P##���Z�HTUU��F�$
�?~���Z�v�����������Z�9��������~�C�����ed2900p���]]]O�>����0��b��d���SW�����>���0IDAT}
uv��1a�bwvk#�goH�r��G������������P�����ceeE&�[ZZrss�NSSS.�[UU���+H�N~#�<'|c�/��9�����{N�@$��O�	D�@�������^�|/M}���X���������{����=���=�����B�s�������s������X�����D"E*N�0������u��������(�F333���������uuu������c��U���1//o�x_���oDD���?����x)))���m411������-$$d���pgSSSPP��pvv�5k�qvv�r��~rrr������}�vcc���O���3fLpp���k@@@aa�����drzzzii)��\�l��}��EY������o���	��>�;e���<055MNNNKK>|8H��dKK���������6&�DJKK�v������C�`�	�]��m��a���_~�������7|���&�;6c��-[�CZ�eQ�N�����b�
###�3���.]
M'�a��I&&&��tvv>}�?���V<U��3g���,X��������$�Dxzz���3����W�����B�?1*��z�jWWWcc���J����w8�@ �@ >)��:����W���?��Czz��#G������:;;��gjj�l�2��hHH��g�LLL"""z{{��[gjjjee���`�pmm��m>���6~�x����B�'%%���������������j���9r����O?����*�3���������x���)))0��F������������o����q����,OO�H�(����������L���������K���s�qr?������������&����111������b�833SZ����������Y����Bl~~~���3g�7n�����5k�t���"SS�#G�\�p*cmmm)))��+�"���������`���xxxddd���JKI�������cbb���u��-���8r������3g���[XX������Y��������b���������6�v��%��q�������S����G�nnn&����6 33s����m�@ ����r�:���1���cJJJWW��������1g������J���������9s��{���������� n(��T	J�U�^���gA�fP�ic�����M�m5�����������T���$I,�D"�X,TU��L��!��uo����S���p���X��7��;w.??����r
������(��NHH
����K�%&&jii��Cb�������{��x~;111���[�nE�(�@ �@ �����zwwwb*`�a�����555�;���b�����R������K�$�xE�������)S���}_X��������'222���@ ��<����w�c��&�����+n~�3@O{��h��Q����D"q}�%�
�d#JJJ��,32��:	���'g����X,V]]����������F �H�=���omm����o���L�@ �@ ���L��#�SRR|||�w��@����i

������wwwwwwdm��!��<��
�FS_<���5��������G�?���.Z����c����������+HJJ���oW�\9P�h�@ �@ �?�0"~|>��Q999��@�
��w�Ko(���@ �@ �@ �@ r�<����@ �@ �@ �@ ���������l �/�@ �@ �@ �@���?��7q"�R^IEND�B`�
#30Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Nagaura, Ryohei (#25)
1 attachment(s)
RE: libpq debug log

Hi,

Because I mistook something about how to reply to e-mails,
my last reply is not reflected in the thread.

Response to Nagaura san's review point, I fixed all his review notes, except for pointing out about psql.
Please see the attached updated patch.

1)
It would be better making the log format the same as the server log format,

I changed date style and added timezone.

2)
It was difficult for me to understand the first line message in the log file.

I changed the message as "The maximum size of this log is 3 Bytes, the parameter 'logminlevel' is set to level2 ".

3)
Under the circumstance that the environment variables "PGLOGDIR" and
"PGLOGSIZE" are set correctly, the log file will also be created when the
user connect the server with "psql".
Does this follow the specification you have thought?
Is there any option to unset only in that session when you want to connect
with "psql"?

By separating session using Tera Term or screen command, you can do what you want.
So I didn't make the code complicated by implementing the option.

4)
Your patch affects the behavior of PQtrace().

Thank you. I fixed.

Regards,
Aya Iwata

Attachments:

v6-0001-libpq-trace-log.patchapplication/octet-stream; name=v6-0001-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index c1d1b6b..ee766fd 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1589,6 +1589,42 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </para>
       </listitem>
     </varlistentry>
+
+     <varlistentry id="libpq-connect-log-dir" xreflabel="logdir">
+      <term><literal>logdir</literal></term>
+      <listitem>
+       <para>
+        Path of directory where log file written. When both <literal>logdir</literal>
+        and <literal>logsize</literal> is set, logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-size" xreflabel="logsize">
+      <term><literal>logsize</literal></term>
+      <listitem>
+       <para>
+        Maximum log size. The unit is megabyte. When the log file size exceeds 
+        to <literal>logsize</literal>, the log is output to another file. 
+        When both <literal>logdir</literal> and <literal>logsize</literal> is set, 
+        logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-min-level" xreflabel="logminlevel">
+      <term><literal>logminlevel</literal></term>
+      <listitem>
+       <para>
+        Level of the log. <literal>level1</literal> and <literal>level2</literal> 
+        can be set, <literal>level1</literal> is the default. 
+        When this parameter is <literal>level1</literal>,it outputs the time 
+        in the function and connection time. <literal>level2</literal> outputs 
+        information on the protocol being exchanged in addition to 
+        <literal>level1</literal> information.
+      </para>
+     </listitem>
+    </varlistentry>
     </variablelist>
    </para>
   </sect2>
@@ -7472,6 +7508,36 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
       linkend="libpq-connect-target-session-attrs"/> connection parameter.
      </para>
     </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGDIR</envar></primary>
+      </indexterm>
+      <envar>PGLOGDIR</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-dir"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGSIZE</envar></primary>
+      </indexterm>
+      <envar>PGLOGSIZE</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-size"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGMINLEVEL</envar></primary>
+      </indexterm>
+      <envar>PGLOGMINLEVEL</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-min-level"/> connection parameter.
+     </para>
+    </listitem>
    </itemizedlist>
   </para>
 
@@ -8300,6 +8366,38 @@ int PQisthreadsafe();
   </para>
  </sect1>
 
+ <sect1 id="libpq-logging">
+  <title>Logging</title>
+
+  <indexterm zone="libpq-logging">
+   <primary>trace log</primary>
+  </indexterm>
+  <indexterm zone="libpq-logging">
+   <primary>logging</primary>
+  </indexterm>
+
+  <para>
+   Libpq trace log can trace information about SQL queries which issued by 
+   the libpq application. This output help determine whether the couse is 
+   on backend side or application side and solve issues with the libpq Driver 
+   when it use. The log without requiring recompile of the libpq application.
+  </para>
+
+  <para>
+   You can activate this log by setting both <parameter>logdir</parameter> and 
+   <parameter>logsize</parameter> of the connection string, or by setting both 
+   the environment variables <envar>PGLOGDIR</envar> and <envar>PGLOGSIZE</envar>.
+   The log trace file is written in a directory setting by <envar>PGLOGDIR</envar> 
+   or <parameter>logdir</parameter>.
+   Information to be output to the log file can be controlled by setting 
+   <parameter>logminlevel</parameter> or <envar>PGLOGMINLEVEL</envar>.
+   The file name is determined as <filename>libpq-%ProcessID-%Y-%m-%d_%H%M%S.log</filename>.
+   If the setting of the file path by the connection string or the environment variable is 
+   incorrect, the log file is not created in the intended location. 
+   The maximum log file size and log level you set is output to the beginning of the file, 
+   so you can check it.
+  </para>
+ </sect1>
 
  <sect1 id="libpq-build">
   <title>Building <application>libpq</application> Programs</title>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index f29202d..c362339 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
 #else
 #define DefaultSSLMode	"disable"
 #endif
+#define DefaultLogMinLevel	"LEVEL1"
 
 /* ----------
  * Definition of the conninfo parameters and their fallback resources.
@@ -325,6 +326,19 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
+	{"logminlevel", "PGLOGMINLEVEL", DefaultLogMinLevel, NULL,
+		"LogMinlevel", "", 7,
+	offsetof(struct pg_conn, logminlevel_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1142,26 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+
+		if(strcmp(conn->logminlevel_str, "level1") == 0)
+			conn->logminlevel = LEVEL1;
+
+		if(strcmp(conn->logminlevel_str, "level2") == 0)
+			conn->logminlevel = LEVEL2;
+
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3716,6 +3750,16 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->logminlevel_str)
+		free(conn->logminlevel_str);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3751,6 +3795,7 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Send connection terminate message to backend");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4153,6 +4198,8 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	traceLog_fprintf(conn, LEVEL1, "Send connection start message to backend");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index ac969e7..6ccd1de 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,8 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	traceLog_fprintf(conn, LEVEL2, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1208,6 +1207,8 @@ fail:
 int
 PQsendQuery(PGconn *conn, const char *query)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery start");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1219,6 +1220,8 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Query: %s",query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1249,6 +1252,9 @@ PQsendQuery(PGconn *conn, const char *query)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery end");
+
 	return 1;
 }
 
@@ -1266,6 +1272,8 @@ PQsendQueryParams(PGconn *conn,
 				  const int *paramFormats,
 				  int resultFormat)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams start");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1283,6 +1291,8 @@ PQsendQueryParams(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams end");
+
 	return PQsendQueryGuts(conn,
 						   command,
 						   "",	/* use unnamed statement */
@@ -1306,6 +1316,8 @@ PQsendPrepare(PGconn *conn,
 			  const char *stmtName, const char *query,
 			  int nParams, const Oid *paramTypes)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare start");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1337,6 +1349,8 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Query: %s ",stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1386,6 +1400,9 @@ PQsendPrepare(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare end");
+
 	return 1;
 
 sendFailed:
@@ -1492,6 +1509,8 @@ PQsendQueryGuts(PGconn *conn,
 {
 	int			i;
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts start");
+
 	/* This isn't gonna work on a 2.0 server */
 	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 	{
@@ -1507,6 +1526,8 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Command: %s \n",stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1638,6 +1659,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts end");
+
 	return 1;
 
 sendFailed:
@@ -1780,6 +1804,8 @@ PQgetResult(PGconn *conn)
 {
 	PGresult   *res;
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult start");
+
 	if (!conn)
 		return NULL;
 
@@ -1820,6 +1846,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -1874,6 +1901,8 @@ PQgetResult(PGconn *conn)
 		}
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult end");
+
 	return res;
 }
 
@@ -2203,6 +2232,8 @@ PQsendDescribePortal(PGconn *conn, const char *portal)
 static int
 PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe start");
+
 	/* Treat null desc_target as empty string */
 	if (!desc_target)
 		desc_target = "";
@@ -2218,6 +2249,8 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Describe type: %c, target: %s \n",desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
@@ -2249,6 +2282,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe end");
+
 	return 1;
 
 sendFailed:
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index e5ef8d4..6a4d4db 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,201 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn,char* filename);
+static void traceLog_fputnbytes(PGconn *conn, int loglevel ,const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime,int type);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime,int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond);
+	else if(type==1)
+	{
+		GetTimezoneInformation(&TimezoneInfo);
+		snprintf(currenttime,TRACELOG_TIME_SIZE,"%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear,localTime.wMonth,localTime.wDay,
+				localTime.wHour,localTime.wMinute,localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+	}
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+	{
+		strftime(timezone, sizeof(timezone), "%Z", tm);
+		snprintf(currenttime, TRACELOG_TIME_SIZE,"%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				1900+ tm->tm_year,1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+	}
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn,char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime,0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir,getpid(),currenttime);
+#endif
+}
+
+/* 
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn,logfilename);
+	conn->traceDebug=fopen(logfilename,"w");
+	fprintf(conn->traceDebug, "The maximum size of this log is %s Bytes, the parameter 'logminlevel' is set to %s\n",
+						 conn->logsize_str, conn->logminlevel_str);
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, PGTraceLogLevel loglevel, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret = 0;
+	char		currenttime[TRACELOG_TIME_SIZE];
+	bool		output_tracelog = true;
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	/* Determin whether to output the log message to the file */
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if(conn->Pfdebug)
+	{
+		/* All protocol messages are set the log level to "LEVEL2". */
+		if(loglevel == LEVEL2)
+			fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+
+	if(conn->traceDebug && output_tracelog)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+			fprintf(conn->traceDebug, "The maximum size of this log is %s Bytes, the parameter 'logminlevel' is set to %s\n",
+					  conn->logsize_str, conn->logminlevel_str);
+		}
+
+		/*
+		 * Select trace log message style. The output style of LEVEL1 or LEVEL2 is
+		 * always selected.
+		 */
+		if(loglevel == LEVEL1)
+		{
+			getCurrentTime(currenttime,1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", currenttime, msgBuf);
+		}
+
+		if(loglevel == LEVEL2)
+		{
+			ret = fprintf(conn->traceDebug, "%s",msgBuf);
+		}
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn,int loglevel, const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+	bool		output_tracelog = true;
+
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug,str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug && output_tracelog)
+	{
+		if(conn->logminlevel < loglevel)
+		{
+			return;
+		}
+
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn,logfilename);
+			conn->traceDebug = fopen(logfilename,"w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug,str,n);
+		fprintf(conn->traceDebug,"\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +295,7 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +310,7 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	traceLog_fprintf(conn, LEVEL2, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +347,8 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	traceLog_fprintf(conn, LEVEL2, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +375,7 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	traceLog_fprintf(conn, LEVEL2, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +387,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,12 +396,8 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -223,15 +413,12 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 int
 pqSkipnchar(size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, conn->inBuffer + conn->inCursor, len);
 
 	conn->inCursor += len;
 
@@ -245,15 +432,12 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf,"To backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -292,8 +476,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +511,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	traceLog_fprintf(conn, LEVEL2, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +730,8 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +767,8 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +857,15 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
 retry3:
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend");
+
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +953,14 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1037,8 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	traceLog_fprintf(conn, LEVEL1, "Start sending message to backend");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1051,8 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	traceLog_fprintf(conn, LEVEL1, "End sending message to backend");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
@@ -963,6 +1158,9 @@ pqFlush(PGconn *conn)
 	if (conn->Pfdebug)
 		fflush(conn->Pfdebug);
 
+	if (conn->traceDebug)
+		fflush(conn->traceDebug);
+
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 97bc98b..b0e6124 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -134,6 +134,16 @@ typedef enum
 	PQPING_NO_ATTEMPT			/* connection not attempted (bad params) */
 } PGPing;
 
+/*
+ * libpq trce log level
+ */
+
+typedef enum
+{
+	LEVEL1,				 /* Time and process (by default) */
+	LEVEL2				 /* Server and Client exchange messages */
+} PGTraceLogLevel;
+
 /* PGconn encapsulates a connection to the backend.
  * The contents of this struct are not supposed to be known to applications.
  */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4a93d8e..1bf62d7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -500,6 +500,13 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logdir;			/* Trace log directory to save log */
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	char	   *logminlevel_str;		/* Trace log level (string)*/
+	int			logsize;		/* Trace log maximum size */
+	PGTraceLogLevel logminlevel;              /* Trace log level( "level1"(default) or "level2") */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -666,6 +673,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, PGTraceLogLevel loglevel, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#31Ramanarayana
raam.soft@gmail.com
In reply to: Iwata, Aya (#30)
Re: libpq debug log

Hi,

I have tested the trace log implementation.Please find my feedback for the
same.

Issues found while testing
---------------------------------
1) If invalid value is given to PGLOGMINLEVEL, empty log file is created
which should not happen.
2) If log file size exceeds the value configured in PGLOGSIZE, new log file
is not getting created.
3) If PGLOGSIZE is greater than 2048 bytes, log file is not created.Is this
expected behaviour?
4) In the log file, an extra new line is present whenever the query is
printed.Is this intentional?
5)Documentation for this feature is having grammatical errors and some
spelling errors which can be looked into.

Feedback in the code
----------------------------------
1) if else statement should be used for checking log levels rather than
multiple if statements
2) Across the code, sufficient space need to be left between parameters in
functions and while using comparison operators
3) In libpq-fe.h in the comments section it should trace rather than trce

Suggestions
----------------------
-> Will it better if the logs of all processes are written to a single log
file with the log message containing the process id rather than one log
file per process?

#32Jamison, Kirk
k.jamison@jp.fujitsu.com
In reply to: Iwata, Aya (#30)
RE: libpq debug log

Hi Iwata-san,

Currently, the patch fails to build according to CF app.
As you know, it has something to do with the misspelling of function.
GetTimezoneInformation --> GetTimeZoneInformation

It sounds more logical to me if there is a parameter that switches on/off the logging
similar to other postgres logs. I suggest trace_log as the parameter name.
Like, this parameter needs to be enabled for logdir, logsize and loglevel, etc. to work.
The default is off.

If you're going to introduce that parameter, then the docs should be updated as well.
i.e. "When <literal>trace_log</literal> is enabled, this parameter..."

How about changing the following parameter names:
logdir --> log_directory
logsize --> log_size
logminlevel --> log_min_level

If it's helpful, you might want to look into how the other postgres logs (i.e. syslogger)
allow setting either absolute or relative path for log directory, and how the parameters
cover some of the comments above by Ramanarayana.

Regards,
Kirk Jamison

#33Robert Haas
robertmhaas@gmail.com
In reply to: Jamison, Kirk (#32)
Re: libpq debug log

On Mon, Feb 18, 2019 at 10:06 PM Jamison, Kirk <k.jamison@jp.fujitsu.com> wrote:

It sounds more logical to me if there is a parameter that switches on/off the logging
similar to other postgres logs. I suggest trace_log as the parameter name.

I'm not really sure that I like the design of this patch in any way.
But leaving that aside, trace_log seems like a poor choice of name,
because it's got two words telling you what kind of thing we are doing
(tracing, logging) and zero words describing the nature of the thing
being traced or logged (the wire protocol communication with the
client). A tracing facility for the planner, or the executor, or any
other part of the system would have an equally good claim on the same
name. That can't be right.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#34Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Ramanarayana (#31)
1 attachment(s)
RE: libpq debug log

Hi Ramanarayana,

Thank you for your review and suggestion. Please see the attached updated patch.

Issues found while testing
1) If invalid value is given to PGLOGMINLEVEL, empty log file is created which should not happen.

Thank you for your test. However in my dev environment, empty log file is not created.
Could you explain more detail about 1)? I will check it again.

2) If log file size exceeds the value configured in PGLOGSIZE, new log file is not getting created.

About 2) (and may be 1) ), perhaps is this something like that?

There are my mistake about first line information of created log file
"The maximum size of this log is %s *Bytes*, the parameter 'logminlevel' is set to %s\n".
- Maximum size is not bytes but megabytes.
- Display logminlevel which set by user. Internally, an invalid value is not set to logminlevel.

So trust the created log file first line info, if you set PGLOGSIZE=1000 as the meaning of "set maximum log size to 1000 Bytes",
a new file was not created even if it exceeds 1000 bytes.
If it is correct, I fixed the comment to output internal setting log maximum size and user setting value.

And if you set PGLOGMINLEVEL to invalid value (ex. "aaa"), it is not set to the parameter; The default value (level1) is set internally.
I fixed first line comment to output notification " if invalid value, level1(default) is set".

3) If PGLOGSIZE is greater than 2048 bytes, log file is not created. Is this expected behavior?

Yes. I limit log file size.

4) In the log file, an extra new line is present whenever the query is printed. Is this intentional?

Thank you, I fixed.

5)Documentation for this feature is having grammatical errors and some spelling errors which can be looked into.

Thank you. I am checking my documentation now. I will fix it.

Feedback in the code

Thank you. I fixed my code issue.

Suggestions

I'll consider that...
Could you explain more about the idea?

Regards,
Aya Iwata

Attachments:

v7-0001-libpq-trace-log.patchapplication/octet-stream; name=v7-0001-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index c1d1b6b..b616100 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1589,6 +1589,43 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
       </para>
       </listitem>
     </varlistentry>
+
+     <varlistentry id="libpq-connect-log-dir" xreflabel="logdir">
+      <term><literal>logdir</literal></term>
+      <listitem>
+       <para>
+        Path of directory where log file written. When both <literal>logdir</literal>
+        and <literal>logsize</literal> is set, logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-size" xreflabel="logsize">
+      <term><literal>logsize</literal></term>
+      <listitem>
+       <para>
+        Maximum log size. The unit is megabyte. When the log file size exceeds 
+        to <literal>logsize</literal>, the log is output to another file. A value 
+        greater than 2048 can not be specified. If specified, logging is deiabled.
+        When both <literal>logdir</literal> and <literal>logsize</literal> is set, 
+        logging is enabled.
+      </para>
+     </listitem>
+    </varlistentry>
+
+     <varlistentry id="libpq-connect-log-min-level" xreflabel="logminlevel">
+      <term><literal>logminlevel</literal></term>
+      <listitem>
+       <para>
+        Level of the log. <literal>level1</literal> and <literal>level2</literal> 
+        can be set, <literal>level1</literal> is the default. 
+        When this parameter is <literal>level1</literal>,it outputs the time 
+        in the function and connection time. <literal>level2</literal> outputs 
+        information on the protocol being exchanged in addition to 
+        <literal>level1</literal> information.
+      </para>
+     </listitem>
+    </varlistentry>
     </variablelist>
    </para>
   </sect2>
@@ -7472,6 +7509,36 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
       linkend="libpq-connect-target-session-attrs"/> connection parameter.
      </para>
     </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGDIR</envar></primary>
+      </indexterm>
+      <envar>PGLOGDIR</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-dir"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGSIZE</envar></primary>
+      </indexterm>
+      <envar>PGLOGSIZE</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-size"/> connection parameter.
+     </para>
+    </listitem>
+
+    <listitem>
+     <para>
+      <indexterm>
+       <primary><envar>PGLOGMINLEVEL</envar></primary>
+      </indexterm>
+      <envar>PGLOGMINLEVEL</envar> behaves the same as the <xref
+      linkend="libpq-connect-log-min-level"/> connection parameter.
+     </para>
+    </listitem>
    </itemizedlist>
   </para>
 
@@ -8300,6 +8367,38 @@ int PQisthreadsafe();
   </para>
  </sect1>
 
+ <sect1 id="libpq-logging">
+  <title>Logging</title>
+
+  <indexterm zone="libpq-logging">
+   <primary>trace log</primary>
+  </indexterm>
+  <indexterm zone="libpq-logging">
+   <primary>logging</primary>
+  </indexterm>
+
+  <para>
+   Libpq trace log can trace information about SQL queries which issued by 
+   the libpq application. This output help determine whether the couse is 
+   on backend side or application side and solve issues with the libpq Driver 
+   when it use. The log without requiring recompile of the libpq application.
+  </para>
+
+  <para>
+   You can activate this log by setting both <parameter>logdir</parameter> and 
+   <parameter>logsize</parameter> of the connection string, or by setting both 
+   the environment variables <envar>PGLOGDIR</envar> and <envar>PGLOGSIZE</envar>.
+   The log trace file is written in a directory setting by <envar>PGLOGDIR</envar> 
+   or <parameter>logdir</parameter>.
+   Information to be output to the log file can be controlled by setting 
+   <parameter>logminlevel</parameter> or <envar>PGLOGMINLEVEL</envar>.
+   The file name is determined as <filename>libpq-%ProcessID-%Y-%m-%d_%H%M%S.log</filename>.
+   If the setting of the file path by the connection string or the environment variable is 
+   incorrect, the log file is not created in the intended location. 
+   The maximum log file size and log level you set is output to the beginning of the file, 
+   so you can check it.
+  </para>
+ </sect1>
 
  <sect1 id="libpq-build">
   <title>Building <application>libpq</application> Programs</title>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index f29202d..a9737b4 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
 #else
 #define DefaultSSLMode	"disable"
 #endif
+#define DefaultLogMinLevel	"LEVEL1"
 
 /* ----------
  * Definition of the conninfo parameters and their fallback resources.
@@ -325,6 +326,19 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	/* libpq trace log options */
+	{"logdir", "PGLOGDIR", NULL, NULL,
+		"Logdir", "", MAXPGPATH - 4,
+	offsetof(struct pg_conn, logdir)},
+
+	{"logsize", "PGLOGSIZE", NULL, NULL,
+		"Logsize", "", 5,
+	offsetof(struct pg_conn, logsize_str)},
+
+	{"logminlevel", "PGLOGMINLEVEL", DefaultLogMinLevel, NULL,
+		"LogMinlevel", "", 7,
+	offsetof(struct pg_conn, logminlevel_str)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1128,6 +1142,29 @@ connectOptions2(PGconn *conn)
 	}
 
 	/*
+	 * If both size and directory of trace log was given,
+	 * initialize a trace log.
+	 */
+	if (conn->logsize_str && conn->logsize_str[0] != '\0')
+		conn->logsize = atoi(conn->logsize_str);
+
+	if (conn->logdir != NULL && conn->logsize > 0 && conn->logsize < 2048)
+	{
+		conn->logsize = conn->logsize * 1024 * 1024;
+
+		if(strcmp(conn->logminlevel_str, "level1") == 0)
+			conn->logminlevel = LEVEL1;
+		else if(strcmp(conn->logminlevel_str, "level2") == 0)
+			conn->logminlevel = LEVEL2;
+
+		/* If invalid value is set as a logminlevel,
+		 * the default value is applied.
+		 */
+
+		initTraceLog(conn);
+	}
+
+	/*
 	 * If password was not given, try to look it up in password file.  Note
 	 * that the result might be different for each host/port pair.
 	 */
@@ -3716,6 +3753,16 @@ freePGconn(PGconn *conn)
 	termPQExpBuffer(&conn->errorMessage);
 	termPQExpBuffer(&conn->workBuffer);
 
+	/* clean up libpq trace log structures */
+	if (conn->logsize_str)
+		free(conn->logsize_str);
+	if (conn->logdir)
+		free(conn->logdir);
+	if (conn->logminlevel_str)
+		free(conn->logminlevel_str);
+	if (conn->traceDebug)
+		fclose(conn->traceDebug);
+
 	free(conn);
 
 #ifdef WIN32
@@ -3751,6 +3798,7 @@ sendTerminateConn(PGconn *conn)
 	 */
 	if (conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Send connection terminate message to backend");
 		/*
 		 * Try to send "close connection" message to backend. Ignore any
 		 * error.
@@ -4153,6 +4201,8 @@ int
 pqPacketSend(PGconn *conn, char pack_type,
 			 const void *buf, size_t buf_len)
 {
+	traceLog_fprintf(conn, LEVEL1, "Send connection start message to backend");
+
 	/* Start the message. */
 	if (pqPutMsgStart(pack_type, true, conn))
 		return STATUS_ERROR;
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index ac969e7..f27c1ae 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -879,7 +879,7 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
 	res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
 	if (res->errMsg)
 	{
-		sprintf(res->errMsg, "%s\n", msgBuf);
+		sprintf(res->errMsg, "NOTICE: %s\n", msgBuf);
 
 		/*
 		 * Pass to receiver, then free it.
@@ -991,9 +991,8 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
+	traceLog_fprintf(conn, LEVEL2, "pqSaveParameterStatus: '%s' = '%s'\n",
+						 name, value);
 
 	/*
 	 * Forget any old information about the parameter
@@ -1208,6 +1207,8 @@ fail:
 int
 PQsendQuery(PGconn *conn, const char *query)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery start");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1219,6 +1220,8 @@ PQsendQuery(PGconn *conn, const char *query)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Query: %s", query);
+
 	/* construct the outgoing Query message */
 	if (pqPutMsgStart('Q', false, conn) < 0 ||
 		pqPuts(query, conn) < 0 ||
@@ -1249,6 +1252,9 @@ PQsendQuery(PGconn *conn, const char *query)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQuery end");
+
 	return 1;
 }
 
@@ -1266,6 +1272,8 @@ PQsendQueryParams(PGconn *conn,
 				  const int *paramFormats,
 				  int resultFormat)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams start");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1283,6 +1291,8 @@ PQsendQueryParams(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryParams end");
+
 	return PQsendQueryGuts(conn,
 						   command,
 						   "",	/* use unnamed statement */
@@ -1306,6 +1316,8 @@ PQsendPrepare(PGconn *conn,
 			  const char *stmtName, const char *query,
 			  int nParams, const Oid *paramTypes)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare start");
+
 	if (!PQsendQueryStart(conn))
 		return 0;
 
@@ -1337,6 +1349,8 @@ PQsendPrepare(PGconn *conn,
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Query: %s ", stmtName, query);
+
 	/* construct the Parse message */
 	if (pqPutMsgStart('P', false, conn) < 0 ||
 		pqPuts(stmtName, conn) < 0 ||
@@ -1386,6 +1400,9 @@ PQsendPrepare(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendPrepare end");
+
 	return 1;
 
 sendFailed:
@@ -1492,6 +1509,8 @@ PQsendQueryGuts(PGconn *conn,
 {
 	int			i;
 
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts start");
+
 	/* This isn't gonna work on a 2.0 server */
 	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 	{
@@ -1507,6 +1526,8 @@ PQsendQueryGuts(PGconn *conn,
 
 	if (command)
 	{
+		traceLog_fprintf(conn, LEVEL1, "Statement name: %s, Command: %s \n", stmtName, command);
+
 		/* construct the Parse message */
 		if (pqPutMsgStart('P', false, conn) < 0 ||
 			pqPuts(stmtName, conn) < 0 ||
@@ -1638,6 +1659,9 @@ PQsendQueryGuts(PGconn *conn,
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendQueryGuts end");
+
 	return 1;
 
 sendFailed:
@@ -1780,6 +1804,8 @@ PQgetResult(PGconn *conn)
 {
 	PGresult   *res;
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult start");
+
 	if (!conn)
 		return NULL;
 
@@ -1820,6 +1846,7 @@ PQgetResult(PGconn *conn)
 
 		/* Parse it. */
 		parseInput(conn);
+
 	}
 
 	/* Return the appropriate thing. */
@@ -1874,6 +1901,8 @@ PQgetResult(PGconn *conn)
 		}
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "PQgetResult end");
+
 	return res;
 }
 
@@ -2203,6 +2232,8 @@ PQsendDescribePortal(PGconn *conn, const char *portal)
 static int
 PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 {
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe start");
+
 	/* Treat null desc_target as empty string */
 	if (!desc_target)
 		desc_target = "";
@@ -2218,6 +2249,8 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 		return 0;
 	}
 
+	traceLog_fprintf(conn, LEVEL1, "Describe type: %c, target: %s \n", desc_type, desc_target);
+
 	/* construct the Describe message */
 	if (pqPutMsgStart('D', false, conn) < 0 ||
 		pqPutc(desc_type, conn) < 0 ||
@@ -2249,6 +2282,9 @@ PQsendDescribe(PGconn *conn, char desc_type, const char *desc_target)
 
 	/* OK, it's launched! */
 	conn->asyncStatus = PGASYNC_BUSY;
+
+	traceLog_fprintf(conn, LEVEL1, "PQsendDescribe end");
+
 	return 1;
 
 sendFailed:
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index e5ef8d4..59f5f4b 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -60,6 +62,198 @@ static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
 
+static void getTraceLogFilename(PGconn *conn, char* filename);
+static void traceLog_fputnbytes(PGconn *conn, int loglevel , const char *head, const char *str, size_t n);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void getCurrentTime(char* currenttime, int type);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * type=0 currenttime formate %Y-%m-%d_%H%M%S
+ * type=1 currenttime formate %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime, int type)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+	if(type==0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d_%02d%02d%02d",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond);
+	else if(type==1)
+	{
+		GetTimeZoneInformation(&TimezoneInfo);
+		snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+	}
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+	if(type == 0)
+		snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d_%02d%02d%02d",
+				1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec);
+	else if(type==1)
+	{
+		strftime(timezone, sizeof(timezone), "%Z", tm);
+		snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+				tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+	}
+#endif
+}
+
+/*
+ * getTraceLogFilename: build trace log file name
+ * The name is libpq-%ProcessID-%Y-%m-%d_%H%M%S.log.
+ */
+static void
+getTraceLogFilename(PGconn *conn, char* filename)
+{
+	char		currenttime[TRACELOG_TIME_SIZE];    /* %Y-%m-%d_%H%M%S */
+	getCurrentTime(currenttime, 0);
+
+#ifdef WIN32
+	snprintf(filename, MAXPGPATH, "%s\\libpq-%d-%s.log", conn->logdir, getpid(), currenttime);
+#else
+	snprintf(filename, MAXPGPATH, "%s/libpq-%d-%s.log", conn->logdir, getpid(), currenttime);
+#endif
+}
+
+/* 
+ * initTraceLog: initialize a trace log file
+ */
+void
+initTraceLog(PGconn *conn)
+{
+	char		logfilename[MAXPGPATH];
+	getTraceLogFilename(conn, logfilename);
+	conn->traceDebug=fopen(logfilename, "w");
+	fprintf(conn->traceDebug, "The maximum size of this file is %s Megabytes (%d Bytes), %s is specified for 'logminlevel' (if invalid value, level1(default) is set)\n",
+			  conn->logsize_str, conn->logsize, conn->logminlevel_str);
+}
+
+/*
+ * traceLog_fprintf: output trace log to file
+ * If PQtrace() is called, PQtrace() is output followed by libpq trace log.
+ */
+void
+traceLog_fprintf(PGconn *conn, PGTraceLogLevel loglevel, const char *fmt,...)
+{
+	char		logfilename[MAXPGPATH];
+	char		msgBuf[MAXPGPATH];
+	va_list		args;
+	int			ret = 0;
+	char		currenttime[TRACELOG_TIME_SIZE];
+	bool		output_tracelog = true;
+
+	va_start(args, fmt);
+	vsnprintf(msgBuf, sizeof(msgBuf), fmt, args);
+	msgBuf[sizeof(msgBuf) - 1] = '\0';
+	va_end(args);
+
+	/* Determin whether to output the log message to the file */
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if(conn->Pfdebug)
+	{
+		/* All protocol messages are set the log level to "LEVEL2". */
+		if(loglevel == LEVEL2)
+			fprintf(conn->Pfdebug, "%s", msgBuf);
+	}
+
+	if(conn->traceDebug && output_tracelog)
+	{
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn, logfilename);
+			conn->traceDebug = fopen(logfilename, "w");
+			if(conn->traceDebug == NULL)
+				return;
+			fprintf(conn->traceDebug, "The maximum size of this file is %s Megabytes (%d Bytes), %s is specified for 'logminlevel' (if invalid value, level1(default) is set)\n",
+					  conn->logsize_str, conn->logsize, conn->logminlevel_str);
+		}
+
+		/*
+		 * Select trace log message style. The output style of LEVEL1 or LEVEL2 is
+		 * always selected.
+		 */
+		if(loglevel == LEVEL1)
+		{
+			getCurrentTime(currenttime, 1);
+			ret = fprintf(conn->traceDebug, "%s  %s\n", currenttime, msgBuf);
+		}
+		else if(loglevel == LEVEL2)
+			ret = fprintf(conn->traceDebug, "%s", msgBuf);
+
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fflush(conn->traceDebug);
+	}
+}
+/*
+ * traceLog_fputnbytes: output trace log to file using fputnbytes()
+ */
+static void
+traceLog_fputnbytes(PGconn *conn, int loglevel, const char *head, const char *str, size_t n)
+{
+	char		logfilename[MAXPGPATH];
+	int			ret;
+	bool		output_tracelog = true;
+
+	if(conn->logminlevel < loglevel)
+		output_tracelog = false;
+
+	if (conn->Pfdebug)
+	{
+		fprintf(conn->Pfdebug, "%s", head);
+		fputnbytes(conn->Pfdebug, str, n);
+		fprintf(conn->Pfdebug, "\n");
+	}
+	else if(conn->traceDebug && output_tracelog)
+	{
+		if(conn->logminlevel < loglevel)
+		{
+			return;
+		}
+
+		if((int)ftell(conn->traceDebug) >= conn->logsize)
+		{
+			fclose(conn->traceDebug);
+			getTraceLogFilename(conn, logfilename);
+			conn->traceDebug = fopen(logfilename, "w");
+			if(conn->traceDebug == NULL)
+				return;
+		}
+		ret = fprintf(conn->traceDebug, "%s", head);
+		if(ret < 0)
+		{
+			fclose(conn->traceDebug);
+			conn->traceDebug = NULL;
+			return;
+		}
+		fputnbytes(conn->traceDebug, str, n);
+		fprintf(conn->traceDebug, "\n");
+		fflush(conn->traceDebug);
+	}
+}
+
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -98,8 +292,7 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend> %c\n", *result);
 
 	return 0;
 }
@@ -114,8 +307,7 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+	traceLog_fprintf(conn, LEVEL2, "To backend> %c\n", c);
 
 	return 0;
 }
@@ -152,9 +344,8 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	traceLog_fprintf(conn, LEVEL2, "From backend> \"%s\"\n",
+						buf->data);
 
 	return 0;
 }
@@ -181,8 +372,7 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+	traceLog_fprintf(conn, LEVEL2, "To backend> \"%s\"\n", s);
 
 	return 0;
 }
@@ -194,6 +384,7 @@ pqPuts(const char *s, PGconn *conn)
 int
 pqGetnchar(char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
@@ -202,12 +393,8 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -223,15 +410,12 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 int
 pqSkipnchar(size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "From backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, conn->inBuffer + conn->inCursor, len);
 
 	conn->inCursor += len;
 
@@ -245,15 +429,12 @@ pqSkipnchar(size_t len, PGconn *conn)
 int
 pqPutnchar(const char *s, size_t len, PGconn *conn)
 {
+	char		buf[100];
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	sprintf(buf, "To backend (%lu)> ", (unsigned long) len);
+	traceLog_fputnbytes(conn, LEVEL2, buf, s, len);
 
 	return 0;
 }
@@ -292,8 +473,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	traceLog_fprintf(conn, LEVEL2, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
 
 	return 0;
 }
@@ -328,8 +508,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+	traceLog_fprintf(conn, LEVEL2, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
 
 	return 0;
 }
@@ -548,9 +727,8 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg %c\n",
+						msg_type ? msg_type : ' ');
 
 	return 0;
 }
@@ -586,9 +764,8 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	traceLog_fprintf(conn, LEVEL2, "To backend> Msg complete, length %u\n",
+						conn->outMsgEnd - conn->outCount);
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
@@ -677,9 +854,15 @@ pqReadData(PGconn *conn)
 	}
 
 	/* OK, try to read some data */
+
 retry3:
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend");
+
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -767,9 +950,14 @@ retry3:
 	 * Still not sure that it's EOF, because some data could have just
 	 * arrived.
 	 */
+
+	traceLog_fprintf(conn, LEVEL1, "Start receiving message from backend");
 retry4:
 	nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
 						  conn->inBufSize - conn->inEnd);
+
+	traceLog_fprintf(conn, LEVEL1, "End receiving message from backend");
+
 	if (nread < 0)
 	{
 		if (SOCK_ERRNO == EINTR)
@@ -846,6 +1034,8 @@ pqSendSome(PGconn *conn, int len)
 	{
 		int			sent;
 
+	traceLog_fprintf(conn, LEVEL1, "Start sending message to backend");
+
 #ifndef WIN32
 		sent = pqsecure_write(conn, ptr, len);
 #else
@@ -858,6 +1048,8 @@ pqSendSome(PGconn *conn, int len)
 		sent = pqsecure_write(conn, ptr, Min(len, 65536));
 #endif
 
+	traceLog_fprintf(conn, LEVEL1, "End sending message to backend");
+
 		if (sent < 0)
 		{
 			/* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble */
@@ -963,6 +1155,9 @@ pqFlush(PGconn *conn)
 	if (conn->Pfdebug)
 		fflush(conn->Pfdebug);
 
+	if (conn->traceDebug)
+		fflush(conn->traceDebug);
+
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 97bc98b..3338067 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -134,6 +134,16 @@ typedef enum
 	PQPING_NO_ATTEMPT			/* connection not attempted (bad params) */
 } PGPing;
 
+/*
+ * libpq trace log level
+ */
+
+typedef enum
+{
+	LEVEL1,				 /* output time and process flow (by default) */
+	LEVEL2				 /* output protocol messages in addition to level1 information */
+} PGTraceLogLevel;
+
 /* PGconn encapsulates a connection to the backend.
  * The contents of this struct are not supposed to be known to applications.
  */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4a93d8e..1bf62d7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -500,6 +500,13 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *logdir;			/* Trace log directory to save log */
+	char	   *logsize_str;		/* Trace log maximum size (string) */
+	char	   *logminlevel_str;		/* Trace log level (string)*/
+	int			logsize;		/* Trace log maximum size */
+	PGTraceLogLevel logminlevel;              /* Trace log level( "level1"(default) or "level2") */
+	FILE	   *traceDebug;			/* Trace log file to write trace info */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -666,6 +673,10 @@ extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
 				 bool got_epipe);
 #endif
 
+/* libpq trace log  */
+extern void initTraceLog(PGconn *conn);
+extern void traceLog_fprintf(PGconn *conn, PGTraceLogLevel loglevel, const char *fmt,...) pg_attribute_printf(3, 4);
+
 /* === SSL === */
 
 /*
#35Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#34)
RE: libpq debug log

Hi Kirk,

Currently, the patch fails to build according to CF app.
As you know, it has something to do with the misspelling of function.
GetTimezoneInformation --> GetTimeZoneInformation

Thank you. I fixed it. Please see my v7 patch.

Regards,
Aya Iwata

#36Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Robert Haas (#33)
RE: libpq debug log

Hi Robert,

I'm not really sure that I like the design of this patch in any way.

Aside from problems with my current documentation which I will fix,
could you explain more detail about the problem of the design?
I would like to improve my current implementation based from feedback.

Regards,
Aya Iwata

#37Jamison, Kirk
k.jamison@jp.fujitsu.com
In reply to: Robert Haas (#33)
RE: libpq debug log

On Wednesday, February 20, 2019 12:56 PM GMT+9, Robert Haas wrote:

On Mon, Feb 18, 2019 at 10:06 PM Jamison, Kirk <k.jamison@jp.fujitsu.com> wrote:

It sounds more logical to me if there is a parameter that switches
on/off the logging similar to other postgres logs. I suggest trace_log as the parameter name.

I'm not really sure that I like the design of this patch in any way.
But leaving that aside, trace_log seems like a poor choice of name,
because it's got two words telling you what kind of thing we are doing
(tracing, logging) and zero words describing the nature of the thing
being traced or logged (the wire protocol communication with the client).
A tracing facility for the planner, or the executor, or any other part
of the system would have an equally good claim on the same name. That
can't be right.

Agreed about the argument with trace_log parameter name. I just shootout
a quick idea. I didn't think about it too deeply, but just thought of a
switch that will enable or disable the feature. So there are
definitely better names other than that. And as you suggested, should
describe specifically what the feature does.

Regards,
Kirk Jamison

#38Robert Haas
robertmhaas@gmail.com
In reply to: Iwata, Aya (#36)
Re: libpq debug log

On Thu, Feb 21, 2019 at 7:52 PM Iwata, Aya <iwata.aya@jp.fujitsu.com> wrote:

I'm not really sure that I like the design of this patch in any way.

Aside from problems with my current documentation which I will fix,
could you explain more detail about the problem of the design?
I would like to improve my current implementation based from feedback.

Well, I believe that what you've got here is something that could,
perhaps, be occasionally useful. However, I don't think it would be
useful to very many people very often, and we'd still have to maintain
the code, so that's not a great situation.

We already have a PQtrace() facility that could be improved, and it
has been previously suggested that you work on improving this facility
rather than inventing something new. I still think that's a good
idea. Instead you've created a second way of producing similar
information, and then coupled it to very specific ideas about how that
information should be logged: it is triggered by new libpq parameters,
there is log rotation logic, etc. Those might not be right for
everyone, and there's no flexibility in the mechanism.

I am not sure that it's a good idea to have facilities that write to
the local filesystem that can be triggered by libpq parameters. Seems
like that might have possible security consequences, or at least annoy
people who want to accept connection strings from users without having
to sanitize them for these sorts of options.

I do sometimes want to know what's going on at the protocol level.
Sometimes it's possible to use wireshark for that (as mentioned
upthread) and when it isn't, the thing I'd really like is for the
command-line clients that already exist have an option to enable
PQtrace without me having to hack the C code. We could go through and
add a long-form command-line option, --libpq-trace, to every
command-line tool we ship, for example. Then we could, as a separate
patch, improve the format of that tracing output. For me that would
be more useful than this.

I don't necessarily think there's anything deeply wrong with this
approach. It's not like your patch will bring about the end of
civilization or anything like that. It just doesn't excite me very
much. And since I agree that we have a problem in this area, I would
ideally like to have a solution to that problem that I feel excited
about.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#39Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#38)
Re: libpq debug log

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Feb 21, 2019 at 7:52 PM Iwata, Aya <iwata.aya@jp.fujitsu.com> wrote:

Aside from problems with my current documentation which I will fix,
could you explain more detail about the problem of the design?

We already have a PQtrace() facility that could be improved, and it
has been previously suggested that you work on improving this facility
rather than inventing something new. I still think that's a good
idea.

Me too. PQtrace as currently constituted might be helpful for debugging
libpq itself, but it's nigh useless for any higher-level purpose. I'd
gladly toss overboard any hypothetical use-case for what it does now,
in favor of having something that dumps stuff at the logical level of
messages. (It's not even very consistent, because somebody did add a
more-or-less-message-level printout to pqSaveParameterStatus, which
I believe is duplicative of what fe-misc.c will print about the
same interaction at the byte level.)

Instead you've created a second way of producing similar
information, and then coupled it to very specific ideas about how that
information should be logged: it is triggered by new libpq parameters,
there is log rotation logic, etc. Those might not be right for
everyone, and there's no flexibility in the mechanism.

Good point. To the extent that people want any of that, they'd probably
want to have application control over it. Maybe, in addition to
PQtrace(FILE *) that'd just dump to a stdio stream, there should be a way
to create a callback similar to a notice-message hook, and let the
application implement such features in a custom callback.

I am not sure that it's a good idea to have facilities that write to
the local filesystem that can be triggered by libpq parameters.

Oy. That seems like a *very* serious objection. I agree with Robert's
thought that it'd be better to insist on application involvement in
enabling trace output.

regards, tom lane

#40Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Tom Lane (#39)
RE: libpq debug log

Hi,

Thank you for your comments and advice.

I'd like to consider your suggestions.
I am planning to change libpq logging like this;

1. Expand PQtrace() facility and improve libpq logging.

2. Prepare "output level". There are 3 type of levels;
- TRADITIONAL : if set, outputs protocol messages
- LEVEL1 : if set, outputs phase and time
- LEVEL2 : if set, outputs both info TRADITIONAL and LEVEL1

3. Add "output phase" information; This is the processing flow. (ex. When PQexec() start and end )

4. Change output method to callback style; Default is stdout, and prepare other callback functions that will be used more frequently.

5. Initialization method;
In current one: PQtrace(PGconn *conn, FILE *stream);
Proposed change: PQtraceEx(PGconn *conn, FILE *stream, PQloggingProcessor callback_func , void *callback_arg, PGLogminlevel level);
PQtrace() can be use as it is to consider compatibility with previous applications,
so I leave PQtrace() and created a new function PQtraceEx().

After discussing the abovementioned, then maybe we can discuss more about enabling trace output and changing the output style.

What do you think? I would appreciate your comments and suggestions.

Regards,
Aya Iwata

#41Kyotaro HORIGUCHI
horiguchi.kyotaro@lab.ntt.co.jp
In reply to: Iwata, Aya (#40)
Re: libpq debug log

Hello.

I came up with some random comments.

At Mon, 4 Mar 2019 08:13:00 +0000, "Iwata, Aya" <iwata.aya@jp.fujitsu.com> wrote in <71E660EB361DF14299875B198D4CE5423DEF1844@g01jpexmbkw25>

Hi,

Thank you for your comments and advice.

I'd like to consider your suggestions.
I am planning to change libpq logging like this;

1. Expand PQtrace() facility and improve libpq logging.

2. Prepare "output level". There are 3 type of levels;
- TRADITIONAL : if set, outputs protocol messages
- LEVEL1 : if set, outputs phase and time
- LEVEL2 : if set, outputs both info TRADITIONAL and LEVEL1

You appear to want to segregate the "traditional" output from
what you want to emit. At least Tom is explicitly suggesting to
throw away the hypothtical use cases for it. You may sort out
what kind of information you/we want to emit as log messages from
scratch:p

You may organize log kinds into hierachical levels like server
log messages or into orthogonal types that are individually
turned on. But it is not necessarily be a parameter of a logging
processor. (mentioned below)

3. Add "output phase" information; This is the processing flow. (ex. When PQexec() start and end )

What is the advantage of it against just two independent messages
like PQexec_start and PQexec_end? (I don't see any advantage.)

4. Change output method to callback style; Default is stdout, and prepare other callback functions that will be used more frequently.

Are you going to make something less-used the default behavior? I
think no one is opposing rich functionality as far as it is
replaceable.

5. Initialization method;
In current one: PQtrace(PGconn *conn, FILE *stream);
Proposed change: PQtraceEx(PGconn *conn, FILE *stream, PQloggingProcessor callback_func , void *callback_arg, PGLogminlevel level);

I'm not sure we should add a new *EX() function. Couldn't we
just change the signature of PQtrace()?

callback_funs seems to be a single function. I think it's better
to have individual function for each message type. Not
callback_func(PQLOG_EXEC_START, param_1, param_2,...) ,but
PQloggingProcessor.PQexec_start(param_1, param_2, ...).

It is because the caller can simply pass values in its own type
to the function without casting or other transformations and
their types are checked statically.

I also think it's better that logger-specific paramters are not
passed in this level. Maybe stream and level are logger-specific
paratmer, which can be combined into callback_arg, or can be
given via an API function.

PQtrace() can be use as it is to consider compatibility with previous applications,
so I leave PQtrace() and created a new function PQtraceEx().

After discussing the abovementioned, then maybe we can discuss more about enabling trace output and changing the output style.

I'm not sure what you mean by "output style" but you can change
everything by replacing the whole callback processor, which may
be a dynamic loaded file which is loaded by the instruction in
both ~/.libpqrc and some API, like PQloadLoggingProcessor()?

What do you think? I would appreciate your comments and suggestions.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#42Robert Haas
robertmhaas@gmail.com
In reply to: Iwata, Aya (#40)
Re: libpq debug log

On Mon, Mar 4, 2019 at 3:13 AM Iwata, Aya <iwata.aya@jp.fujitsu.com> wrote:

2. Prepare "output level". There are 3 type of levels;
- TRADITIONAL : if set, outputs protocol messages
- LEVEL1 : if set, outputs phase and time
- LEVEL2 : if set, outputs both info TRADITIONAL and LEVEL1

I am not impressed by this proposal. I think what we should be
focusing on here is how to clearly display the contents of a message.
I think we should be looking for a way to display each message on a
single line in a way that indicates the data types of the constituent
fields. For example, here's a DataRow message as output by PQtrace
today:

From backend> D
From backend (#4)> 42
From backend (#2)> 4
From backend (#4)> 6
From backend (6)> public
From backend (#4)> 4
From backend (4)> tab1
From backend (#4)> 5
From backend (5)> table
From backend (#4)> 5
From backend (5)> rhaas

What I'd like to see for a case like this is something like:

<<< 'D' 42 #4 6 'public' 4 'tab1' 5 'table' 5 'rhaas'

And here's a RowDescription message today:

From backend> T
From backend (#4)> 101
From backend (#2)> 4
From backend> "Schema"
From backend (#4)> 2615
From backend (#2)> 2
From backend (#4)> 19
From backend (#2)> 64
From backend (#4)> -1
From backend (#2)> 0
From backend> "Name"
From backend (#4)> 1259
From backend (#2)> 2
From backend (#4)> 19
From backend (#2)> 64
From backend (#4)> -1
From backend (#2)> 0
From backend> "Type"
From backend (#4)> 0
From backend (#2)> 0
From backend (#4)> 25
From backend (#2)> 65535
From backend (#4)> -1
From backend (#2)> 0
From backend> "Owner"
From backend (#4)> 0
From backend (#2)> 0
From backend (#4)> 19
From backend (#2)> 64
From backend (#4)> -1
From backend (#2)> 0

And I propose that it should look like this:

<<< 'T' 101 4 "Schema" 2615 #2 19 #64 -1 #0 "Name" 1259 #2 19 #64 -1
#0 "Owner" 0 #0 19 #64 -1 #0

The basic idea being:

- Each line is a whole message.
- The line begins with <<< for a message received and >>> for a message sent.
- Strings in single quotes are those sent/received as a fixed number of bytes.
- Strings in double quotes are those sent/received as a string.
- 4-byte integers are printed unadorned.
- 2-byte integers are prefixed by #.
- I guess 1-byte integers would need some other prefix, maybe @ or ##.

Now if we want to timestamp those lines too, that'd be fine:

2019-03-04 21:33:39.338 EST <<< 'T' 101 4 "Schema" 2615 #2 19 #64 -1
#0 "Name" 1259 #2 19 #64 -1 #0 "Owner" 0 #0 19 #64 -1 #0
2019-03-04 21:33:39.342 EST <<< 'D' 42 #4 6 'public' 4 'tab1' 5
'table' 5 'rhaas'

But I still don't really see a need for different levels or whatever.
I mean, you either want a dump of all of the protocol traffic, or you
don't, I think. Or maybe I am confused as to what the goal of all
this really is.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#43Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#42)
Re: libpq debug log

Robert Haas <robertmhaas@gmail.com> writes:

The basic idea being:

- Each line is a whole message.
- The line begins with <<< for a message received and >>> for a message sent.

+1, though do we really need to repeat the direction marker thrice?

- Strings in single quotes are those sent/received as a fixed number of bytes.
- Strings in double quotes are those sent/received as a string.
- 4-byte integers are printed unadorned.
- 2-byte integers are prefixed by #.
- I guess 1-byte integers would need some other prefix, maybe @ or ##.

I doubt that anybody gives a fig for those distinctions, except when
they're writing actual code that speaks the protocol --- and I do not
think that that's the target use-case. So strings and integers seem
like plenty. I'd also suggest that just because the protocol has
single-letter codes for message types doesn't mean that average users
have memorized those codes; and that framing data like the message
length is of no interest.

In short, rather than

<<< 'T' 101 4 "Schema" 2615 #2 19 #64 -1 #0 "Name" 1259 #2 19 #64 -1 #0 "Owner" 0 #0 19 #64 -1 #0

I'd envision something more like

< RowDescription "Schema" 2615 2 19 64 -1 0 "Name" 1259 2 19 64 -1 0 "Owner" 0 0 19 64 -1 0

But I still don't really see a need for different levels or whatever.
I mean, you either want a dump of all of the protocol traffic, or you
don't, I think. Or maybe I am confused as to what the goal of all
this really is.

Yeah, me too. But a lot of this detail would only be useful if you
were trying to diagnose something like a discrepancy between the server
and libpq as to the width of some field. And the number of users for
that can be counted without running out of fingers. I think what would
be of use for a trace facility is as high-level a view as possible of
the message contents.

Or, in other words: a large part of the problem with the existing PQtrace
facility is that it *was* designed to help debug libpq itself, and that
use-case is no longer very interesting. We should assume that the library
knows how to parse protocol messages.

regards, tom lane

#44Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#43)
Re: libpq debug log

On Mon, Mar 4, 2019 at 10:25 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

The basic idea being:
- Each line is a whole message.
- The line begins with <<< for a message received and >>> for a message sent.

+1, though do we really need to repeat the direction marker thrice?

Perhaps not.

- Strings in single quotes are those sent/received as a fixed number of bytes.
- Strings in double quotes are those sent/received as a string.
- 4-byte integers are printed unadorned.
- 2-byte integers are prefixed by #.
- I guess 1-byte integers would need some other prefix, maybe @ or ##.

I doubt that anybody gives a fig for those distinctions, except when
they're writing actual code that speaks the protocol --- and I do not
think that that's the target use-case. So strings and integers seem
like plenty. I'd also suggest that just because the protocol has
single-letter codes for message types doesn't mean that average users
have memorized those codes; and that framing data like the message
length is of no interest.

I don't agree with that. For one thing, I'm someone, and I give a fig.

I would put it this way: with a very small amount of additional
notation it's possible to preserve the level of detail that we have
currently, and I think that's worth it. Your proposed format for the
sample message I showed is very slightly shorter, which will almost
certainly not matter to anyone, but it leaves some slight ambiguity
about what was happening at the protocol level, which might. If you
don't care, the additional detail in my proposed format is easy enough
to ignore.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#45Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Kyotaro HORIGUCHI (#41)
RE: libpq debug log

Hi Horiguchi-san,

Thank you for your reply and suggestions.

1. Expand PQtrace() facility and improve libpq logging.

2. Prepare "output level". There are 3 type of levels;
- TRADITIONAL : if set, outputs protocol messages
- LEVEL1 : if set, outputs phase and time
- LEVEL2 : if set, outputs both info TRADITIONAL and LEVEL1

You appear to want to segregate the "traditional" output from what you want
to emit. At least Tom is explicitly suggesting to throw away the hypothtical
use cases for it. You may sort out what kind of information you/we want to
emit as log messages from scratch:p

You may organize log kinds into hierachical levels like server log messages
or into orthogonal types that are individually turned on. But it is not
necessarily be a parameter of a logging processor. (mentioned below)

It is intended for old application users who use PQtrace() expect the existing/default/traditional log output style.
That’s why I separated other information(ex. phase and timestamp).
But since you mentioned the level is not necessary,
I will follow your advice and include those information in the reformatted PQtrace().

3. Add "output phase" information; This is the processing flow. (ex.
When PQexec() start and end )

What is the advantage of it against just two independent messages like
PQexec_start and PQexec_end? (I don't see any advantage.)

I think the purpose of this logging improvement is to make it useful for analysis at performance deterioration.
When query delay happens, we want to know from which side(server or client) is the cause of it,
and then people want to know which process takes time.
I think the phase and time information are useful for diagnosis.
For example, when command processing function (ex. PQexec()) etc. start/end
and when client receive/send protocol messages.
/*my intended output here */

4. Change output method to callback style; Default is stdout, and prepare

other callback functions that will be used more frequently.

Are you going to make something less-used the default behavior? I think no
one is opposing rich functionality as far as it is replaceable.

I am sorry, my explanation was not clear.
I just wanted to say I intend to provide several output method functions which users likely need.
ex. output to stdout, or output to file, or output to log directory.

5. Initialization method;
In current one: PQtrace(PGconn *conn, FILE *stream); Proposed change:
PQtraceEx(PGconn *conn, FILE *stream, PQloggingProcessor callback_func
, void *callback_arg, PGLogminlevel level);

I'm not sure we should add a new *EX() function. Couldn't we just change the
signature of PQtrace()?

I intended to add new *EX() function for compatibility purposes specially for old version of libpq.
I would like to avoid making changes to old applications for this.
But if we insist on changing the PQtrace() itself, then I will follow your advice.

callback_funs seems to be a single function. I think it's better to have
individual function for each message type. Not
callback_func(PQLOG_EXEC_START, param_1, param_2,...) ,but
PQloggingProcessor.PQexec_start(param_1, param_2, ...).

It is because the caller can simply pass values in its own type to the function
without casting or other transformations and their types are checked
statically.

I also think it's better that logger-specific paramters are not passed in
this level. Maybe stream and level are logger-specific paratmer, which can
be combined into callback_arg, or can be given via an API function.

Thank you for your advice. I will consider it.

Regards,
Aya Iwata

#46Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Tom Lane (#43)
RE: libpq debug log

Hi everyone,

I appreciate all the helpful advice.

I agree to display message more clearly. I will follow your advice.

I would like to add timestamp per line and when command processing function start/end.
I think it is useful to know the application process start/end for diagnosis.
So I will implement like this;

2019-03-03 07:24:54.142 UTC PQgetResult start
2019-03-03 07:24:54.143 UTC < 'T' 35 1 "set_config" 0 #0 25 #65535 -1 #0
2019-03-03 07:24:54.144 UTC PQgetResult end

But I still don't really see a need for different levels or whatever.
I mean, you either want a dump of all of the protocol traffic, or you
don't, I think. Or maybe I am confused as to what the goal of all
this really is.

Yeah, me too. But a lot of this detail would only be useful if you were trying
to diagnose something like a discrepancy between the server and libpq as to
the width of some field. And the number of users for that can be counted
without running out of fingers. I think what would be of use for a trace
facility is as high-level a view as possible of the message contents.

Or, in other words: a large part of the problem with the existing PQtrace
facility is that it *was* designed to help debug libpq itself, and that
use-case is no longer very interesting. We should assume that the library
knows how to parse protocol messages.

Since I explained the reason in the previous email, I am copy-pasting it again here.

I think the purpose of the leveling is to provide an optional information for the user,
which is useful for diagnosis during the performance deterioration.
When query delay happens, we want to know from which side(server or client) is the cause of it,
and then people want to know which process takes time.
I think the phase and time information are useful for diagnosis.
For example, when command processing function (ex. PQexec()) etc. start/end
and when client receive/send protocol messages.

So is it alright to add these information to the new/proposed PQtrace() default output?

Regards,
Aya Iwata

#47David Steele
david@pgmasters.net
In reply to: Iwata, Aya (#46)
Re: RE: libpq debug log

On 3/5/19 11:48 AM, Iwata, Aya wrote:

So is it alright to add these information to the new/proposed PQtrace() default output?

I agree with Andres [1]/messages/by-id/raw/20190214203752.t4hl574k6jlu4t25@alap3.anarazel.de that it's not very clear where this patch is
going and we should push the target to PG13.

Regards,
--
-David
david@pgmasters.net

[1]: /messages/by-id/raw/20190214203752.t4hl574k6jlu4t25@alap3.anarazel.de
/messages/by-id/raw/20190214203752.t4hl574k6jlu4t25@alap3.anarazel.de

#48David Steele
david@pgmasters.net
In reply to: David Steele (#47)
Re: Re: RE: libpq debug log

On 3/5/19 5:28 PM, David Steele wrote:

On 3/5/19 11:48 AM, Iwata, Aya wrote:

So is it alright to add these information to the new/proposed
PQtrace() default output?

I agree with Andres [1] that it's not very clear where this patch is
going and we should push the target to PG13.

Hearing no opinions to the contrary, I have set the target version to PG13.

Regards,
--
-David
david@pgmasters.net

#49Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#46)
1 attachment(s)
RE: libpq debug log

Hi,

The basic idea being:

- Each line is a whole message.
- The line begins with <<< for a message received and >>> for a message sent.
- Strings in single quotes are those sent/received as a fixed number of bytes.
- Strings in double quotes are those sent/received as a string.
- 4-byte integers are printed unadorned.
- 2-byte integers are prefixed by #.
- I guess 1-byte integers would need some other prefix, maybe @ or ##.

I created v1 patch to improve PQtrace(); output log message in one line.
Please find my attached patch.

Log output examples;

In current PQtrace log:

To backend> Msg Q
To backend> "SELECT pg_catalog.set_config('search_path', '', false)"
To backend> Msg complete, length 60

I changed like this:

2019-04-04 02:39:51.488 UTC > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"

In current PQtrace log:

From backend> T
From backend (#4)> 35
From backend (#2)> 1
From backend> "set_config"
From backend (#4)> 0
From backend (#2)> 0
From backend (#4)> 25
From backend (#2)> 65535
From backend (#4)> -1
From backend (#2)> 0

I changed like this:

2019-04-04 02:39:51.489 UTC < RowDescription 35 #1 "set_config" 0 #0 25 #65535 -1 #0

I would like to add timestamp per line and when command processing function
start/end.
I think it is useful to know the application process start/end for diagnosis.
So I will implement like this;

2019-03-03 07:24:54.142 UTC PQgetResult start
2019-03-03 07:24:54.143 UTC < 'T' 35 1 "set_config" 0 #0 25 #65535 -1 #0
2019-03-03 07:24:54.144 UTC PQgetResult end

I would like to add this in next patch if there are not any disagreement.

Regards,
Aya Iwata

Attachments:

v1-0001-libpq-PQtrace-output_one_line.patchapplication/octet-stream; name=v1-0001-libpq-PQtrace-output_one_line.patchDownload
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ea4c9d2..66a9eb3 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,12 +55,335 @@
 #include "port/pg_bswap.h"
 #include "pg_config_paths.h"
 
+/* protocol message name */
+static char *command_text_b[] = {
+	/* 0 */ 0,
+	/* 1 */ "ParseComplete",
+	/* 2 */ "BindComplete",
+	/* 3 */ "CloseComplete",
+	/* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ "NotificationResponse",
+	/* B */ 0,
+	/* C */ "CommandComplete",
+	/* D */ "DataRow",
+	/* E */ "ErrorResponse",
+	/* F */ 0,
+	/* G */ "CopyInResponse",
+	/* H */ "CopyOutResponse",
+	/* I */ "EmptyQueryResponse",
+	/* J */ 0,
+	/* K */ "BackendKeyData",
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ "NoticeResponse",
+	/* O */ 0,
+	/* P */ 0,
+	/* Q */ 0,
+	/* R */ "Authentication",
+	/* S */ "ParameterStatus",
+	/* T */ "RowDescription",
+	/* U */ 0,
+	/* V */ "FunctionCallResponse",
+	/* W */ "CopyBothResponse",
+	/* X */ 0,
+	/* Y */ 0,
+	/* Z */ "ReadyForQuery",
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* \x60 - \x6d*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* n */ "NoData",
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ "PortalSuspended",
+	/* t */ "ParameterDescription",
+	/* u */ 0,
+	/* v */ "NegotiateProtocolVersion",
+	/* w */ 0,
+	/* x */ 0,
+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_B_MAX (sizeof(command_text_b) / sizeof(*command_text_b))
+
+static char *command_text_f[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ 0,
+	/* B */ "Bind",
+	/* C */ "Close",
+	/* D */ "Describe",
+	/* E */ "Execute",
+	/* F */ "FunctionCall",
+	/* G */ 0,
+	/* H */ "Flush",
+	/* I */ 0,
+	/* J */ 0,
+	/* K */ 0,
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ 0,
+	/* O */ 0,
+	/* P */ "Parse",
+	/* Q */ "Query",
+	/* R */ 0,
+	/* S */ "Sync",
+	/* T */ 0,
+	/* U */ 0,
+	/* V */ 0,
+	/* W */ 0,
+	/* X */ "Terminate",
+	/* Y */ 0,
+	/* Z */ 0,
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ 0,
+	/* d */ 0,
+	/* e */ 0,
+	/* f */ "CopyFail",
+	/* g */ 0,
+	/* h */ 0,
+	/* i */ 0,
+	/* j */ 0,
+	/* k */ 0,
+	/* l */ 0,
+	/* m */ 0,
+	/* n */ 0,
+	/* o */ 0,
+	/* p */ "AuthnenticationResponse",
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ 0,
+	/* t */ 0,
+	/* u */ 0,
+	/* v */ 0,
+	/* w */ 0,
+	/* x */ 0,
+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_F_MAX (sizeof(command_text_f) / sizeof(*command_text_f))
+
+static char *command_text_bf[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x40 - \x4f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x50 - \x5f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ "CopyDone",
+	/* d */ "CopyData",
+	/* e */ 0,
+	/* f */ 0,
+	/* g */ 0,
+	/* h */ 0,
+	/* i */ 0,
+	/* j */ 0,
+	/* k */ 0,
+	/* l */ 0,
+	/* m */ 0,
+	/* n */ 0,
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ 0,
+	/* t */ 0,
+	/* u */ 0,
+	/* v */ 0,
+	/* w */ 0,
+	/* x */ 0,
+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_BF_MAX (sizeof(command_text_bf) / sizeof(*command_text_bf))
+
+/*
+ * static state needed by logging output; display
+ * contents of message in one line.
+ */
+LoggingMsg logging_message;
+Frontend_Entry frontend_entry[MAXPGPATH];
+static int		entry;
+
 
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void 	message_Byte1(PGconn *conn, char v, CommunicationDirection direction);
+static void 	message_String(PGconn *conn, const char *v, int length,
+				CommunicationDirection direction);
+static void 	message_nchar(PGconn *conn, const char *v, int length,
+				CommunicationDirection direction);
+static void 	message_Int(PGconn *conn, int v, int length, CommunicationDirection direct);
+static void 	getCurrentTime(char* currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/* Get Protocol message text from byte1 identifier */
+static char *
+message_get_command_text(unsigned char c, CommunicationDirection direction)
+{
+	char *text = 0;
+
+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		text = command_text_bf[c];
+		if (text)
+			return text;
+	}
+
+	if (direction == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		text = command_text_b[c];
+		if (text)
+			return text;
+	}
+
+	if (direction == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)
+	{
+		text = command_text_f[c];
+		if (text)
+			return text;
+	}
+
+	return "UnknownCommand";
+}
+
+/* Initializing logging message */
+static void
+initialize_message_log(void) {
+	logging_message.state = LOG_COMMAND;
+	logging_message.length = 0;
+}
+
+/* Output a message the protocol is invalid */
+static void
+message_invalid_protocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	logging_message.state = LOG_COMMAND;
+}
+
+/*
+ * Check whether message formats is complete. If so,
+ * break the line.
+ */
+void
+message_state_transaction(int size, PGconn *conn)
+{
+	logging_message.length -= size;
+	if (logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		initialize_message_log();
+	}
+}
+
+/*
+ * Remember messages that frontend sends.
+ */
+static void message_entry(PGconn *conn, Type type, int length)
+{
+	frontend_entry[entry].type  = type;
+	frontend_entry[entry].message_addr  = conn->outMsgEnd - length;
+	frontend_entry[entry].message_length  = length;
+	entry++;
+}
+
+/*
+ * After output frontend message length, output organized frontend message.
+ */
+static void put_frontend_entry(PGconn *conn)
+{
+	int 		i;
+	int 		message_addr;
+	int 		length;
+
+	char 		message;
+	int		result = 0;
+
+	for (i = 0; i < entry; i++)
+	{
+		message_addr = frontend_entry[i].message_addr;
+		length = frontend_entry[i].message_length;
+
+		switch (frontend_entry[i].type)
+		{
+			case BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				message_Byte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case STRING:
+				message_String(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case NCHAR:
+				message_nchar(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case INT16:
+				memcpy(&result, conn->outBuffer + message_addr, length);
+				message_Int(conn, result, 2, FROM_FRONTEND);
+				break;
+
+			case INT32:
+				memcpy(&result, conn->outBuffer + message_addr, length);
+				message_Int(conn, result, 4, FROM_FRONTEND);
+				break;
+		}
+	}
+	entry = 0;
+	initialize_message_log();
+}
+
+/*
+ * getCurrentTime: get current time for trace log output
+ *
+ * currenttime formate is %Y/%m/%d %H:%M:%S.%Milliseconds
+ */
+static void
+getCurrentTime(char* currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -99,12 +424,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		message_Byte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -115,11 +439,41 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		message_entry(conn, BYTE1, 1);
 
 	return 0;
 }
 
+static void
+message_Byte1(PGconn *conn, char v, CommunicationDirection direction)
+{
+	char *command_text;
+	char *direction_text = direction == FROM_BACKEND ? "<" : ">";
+	char current_time[TRACELOG_TIME_SIZE];
+
+	switch (logging_message.state) {
+		case LOG_COMMAND:
+			command_text = message_get_command_text((unsigned char) v, direction);
+			getCurrentTime(current_time);
+			fprintf(conn->Pfdebug, "%s %s %s ", current_time, direction_text, command_text);
+			logging_message.state = LOG_LENGTH;
+			logging_message.command = v;
+			entry = 0;
+			break;
+
+		case LOG_BODY:
+			fprintf(conn->Pfdebug, "%c ", v);
+			message_state_transaction(sizeof(v), conn);
+			break;
+
+		default:
+			message_invalid_protocol(conn);
+			break;
+	}
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -153,8 +507,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		message_String(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -182,11 +535,30 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		message_entry(conn, STRING, strlen(s) + 1);
 
 	return 0;
 }
 
+static void
+message_String(PGconn *conn, const char *v, int length, CommunicationDirection direction)
+{
+	if(length < 0)
+		length = strlen(v) + 1;
+
+	switch (logging_message.state)
+	{
+		case LOG_BODY:
+			fprintf(conn->Pfdebug, "\"%s\" ", v);
+			message_state_transaction(length, conn);
+			break;
+
+		default:
+			message_invalid_protocol(conn);
+			break;
+	}
+}
+
 /*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
@@ -203,11 +575,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		message_nchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -227,11 +595,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		message_nchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -249,15 +613,29 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		message_entry(conn, NCHAR, len);
 
 	return 0;
 }
 
+static void
+message_nchar(PGconn *conn, const char *v, int length, CommunicationDirection direction)
+{
+	switch (logging_message.state)
+	{
+		case LOG_BODY:
+			fprintf(conn->Pfdebug, "\'");
+			fputnbytes(conn->Pfdebug, v, length);
+			fprintf(conn->Pfdebug, "\' ");
+			message_state_transaction(length, conn);
+			break;
+
+		default:
+			message_invalid_protocol(conn);
+			break;
+	}
+}
+
 /*
  * pqGetInt
  *	read a 2 or 4 byte integer and convert from network byte order
@@ -293,7 +671,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		message_Int(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -315,11 +693,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				message_entry(conn, INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				message_entry(conn, INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -327,11 +709,32 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+static void
+message_Int(PGconn *conn, int v, int length, CommunicationDirection direct)
+{
+	char *prefix = length == 4 ? "" : "#";
 
-	return 0;
+	switch (logging_message.state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d ", v);
+			logging_message.length = v - length;
+			logging_message.state = LOG_BODY;
+			message_state_transaction(0, conn);
+			break;
+
+		case LOG_BODY:
+			fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+			message_state_transaction(length, conn);
+			break;
+
+		default:
+			message_invalid_protocol(conn);
+			break;
+	}
 }
 
 /*
@@ -549,8 +952,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		message_Byte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -587,8 +989,12 @@ int
 pqPutMsgEnd(PGconn *conn)
 {
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
+	{
+		/* Get protocol message length (including first Byte1) */
+		int length = conn->outMsgEnd - conn->outCount;
+		message_Int(conn, length - 1, 4, FROM_FRONTEND);
+		put_frontend_entry(conn);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index ec51fee..c1a9356 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -125,6 +125,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				message_state_transaction(msgLength, conn);
 			return;
 		}
 
@@ -158,7 +161,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					message_state_transaction(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 84222f2..9366bbf 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -545,6 +545,44 @@ extern char *const pgresStatus[];
 
 #endif							/* USE_SSL */
 
+/* Logging message */
+typedef enum CommunicationDirection
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} CommunicationDirection;
+
+typedef enum State
+{
+	LOG_COMMAND,
+	LOG_LENGTH,
+	LOG_BODY
+} State;
+
+typedef struct _LoggingMsg
+{
+	State state;
+	int length;
+	char command;
+} LoggingMsg;
+
+/* Organizing frontend message.  */
+typedef enum Type
+{
+	BYTE1,
+	STRING,
+	NCHAR,
+	INT16,
+	INT32
+} Type;
+
+typedef struct _Frontend_Entry
+{
+	Type type;
+	int message_addr;
+	int message_length;
+} Frontend_Entry;
+
 /* ----------------
  * Internal functions of libpq
  * Functions declared here need to be visible across files of libpq,
@@ -655,6 +693,7 @@ extern int pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 			time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void	message_state_transaction(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#50Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#49)
1 attachment(s)
RE: libpq debug log

Hi,

I update patch to improve PQtrace(); output log message in one line.
Please find my attached patch.

How it changed:

The basic idea being:

- Each line is a whole message.
- The line begins with <<< for a message received and >>> for a message

sent.

- Strings in single quotes are those sent/received as a fixed number of

bytes.

- Strings in double quotes are those sent/received as a string.
- 4-byte integers are printed unadorned.
- 2-byte integers are prefixed by #.
- I guess 1-byte integers would need some other prefix, maybe @ or ##.

New log output examples:
The message sent from frontend is like this;
2019-04-04 02:39:51.488 UTC > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"

The message sent from backend is like this;
2019-04-04 02:39:51.489 UTC < RowDescription 35 #1 "set_config" 0 #0 25 #65535 -1 #0

Regards,
Aya Iwata

Attachments:

v2-0001-libpq-PQtrace-output_one_line.patchapplication/octet-stream; name=v2-0001-libpq-PQtrace-output_one_line.patchDownload
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ea4c9d2..20cfef3 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,12 +55,377 @@
 #include "port/pg_bswap.h"
 #include "pg_config_paths.h"
 
+/* protocol message name */
+static char *command_text_b[] = {
+	/* 0 */ 0,
+	/* 1 */ "ParseComplete",
+	/* 2 */ "BindComplete",
+	/* 3 */ "CloseComplete",
+	/* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ "NotificationResponse",
+	/* B */ 0,
+	/* C */ "CommandComplete",
+	/* D */ "DataRow",
+	/* E */ "ErrorResponse",
+	/* F */ 0,
+	/* G */ "CopyInResponse",
+	/* H */ "CopyOutResponse",
+	/* I */ "EmptyQueryResponse",
+	/* J */ 0,
+	/* K */ "BackendKeyData",
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ "NoticeResponse",
+	/* O */ 0,
+	/* P */ 0,
+	/* Q */ 0,
+	/* R */ "Authentication",
+	/* S */ "ParameterStatus",
+	/* T */ "RowDescription",
+	/* U */ 0,
+	/* V */ "FunctionCallResponse",
+	/* W */ "CopyBothResponse",
+	/* X */ 0,
+	/* Y */ 0,
+	/* Z */ "ReadyForQuery",
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* \x60 - \x6d*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* n */ "NoData",
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ "PortalSuspended",
+	/* t */ "ParameterDescription",
+	/* u */ 0,
+	/* v */ "NegotiateProtocolVersion",
+	/* w */ 0,
+	/* x */ 0,
+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_B_MAX (sizeof(command_text_b) / sizeof(*command_text_b))
+
+static char *command_text_f[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ 0,
+	/* B */ "Bind",
+	/* C */ "Close",
+	/* D */ "Describe",
+	/* E */ "Execute",
+	/* F */ "FunctionCall",
+	/* G */ 0,
+	/* H */ "Flush",
+	/* I */ 0,
+	/* J */ 0,
+	/* K */ 0,
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ 0,
+	/* O */ 0,
+	/* P */ "Parse",
+	/* Q */ "Query",
+	/* R */ 0,
+	/* S */ "Sync",
+	/* T */ 0,
+	/* U */ 0,
+	/* V */ 0,
+	/* W */ 0,
+	/* X */ "Terminate",
+	/* Y */ 0,
+	/* Z */ 0,
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ 0,
+	/* d */ 0,
+	/* e */ 0,
+	/* f */ "CopyFail",
+	/* g */ 0,
+	/* h */ 0,
+	/* i */ 0,
+	/* j */ 0,
+	/* k */ 0,
+	/* l */ 0,
+	/* m */ 0,
+	/* n */ 0,
+	/* o */ 0,
+	/* p */ "AuthnenticationResponse",
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ 0,
+	/* t */ 0,
+	/* u */ 0,
+	/* v */ 0,
+	/* w */ 0,
+	/* x */ 0,
+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_F_MAX (sizeof(command_text_f) / sizeof(*command_text_f))
+
+static char *command_text_bf[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x40 - \x4f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x50 - \x5f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ "CopyDone",
+	/* d */ "CopyData",
+	/* e */ 0,
+	/* f */ 0,
+	/* g */ 0,
+	/* h */ 0,
+	/* i */ 0,
+	/* j */ 0,
+	/* k */ 0,
+	/* l */ 0,
+	/* m */ 0,
+	/* n */ 0,
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ 0,
+	/* t */ 0,
+	/* u */ 0,
+	/* v */ 0,
+	/* w */ 0,
+	/* x */ 0,
+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_BF_MAX (sizeof(command_text_bf) / sizeof(*command_text_bf))
+
 
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void	fputnbytes(FILE *f, const char *str, size_t n);
+static void 	message_Byte1(PGconn *conn, char v, CommunicationDirection direction);
+static void 	message_String(PGconn *conn, const char *v, int length,
+				CommunicationDirection direction);
+static void 	message_nchar(PGconn *conn, const char *v, int length,
+				CommunicationDirection direction);
+static void 	message_Int(PGconn *conn, int v, int length, CommunicationDirection direct);
+static void 	getCurrentTime(char* currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * message_get_command_text:
+ * 	Get Protocol message text from byte1 identifier
+ */
+static char *
+message_get_command_text(unsigned char c, CommunicationDirection direction)
+{
+	char *text = 0;
+
+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		text = command_text_bf[c];
+		if (text)
+			return text;
+	}
+
+	if (direction == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		text = command_text_b[c];
+		if (text)
+			return text;
+	}
+
+	if (direction == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)
+	{
+		text = command_text_f[c];
+		if (text)
+			return text;
+	}
+
+	return "UnknownCommand";
+}
+
+/* initialize_message_log: Initializing logging message */
+static void
+initialize_message_log(PGconn *conn) {
+	conn->logging_message.state = LOG_COMMAND;
+	conn->logging_message.length = 0;
+}
+
+/* message_invalid_protocol: Output a message the protocol is invalid */
+static void
+message_invalid_protocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_COMMAND;
+}
+
+/*
+ * message_state_transaction:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+message_state_transaction(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		initialize_message_log(conn);
+	}
+}
+
+/*
+ * message_entry: Remember messages that frontend sends.
+ */
+static void message_entry(PGconn *conn, Type type, int length)
+{
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->frontend_entry[conn->entry].type  = type;
+		conn->frontend_entry[conn->entry].message_addr  = conn->outMsgEnd - length;
+		conn->frontend_entry[conn->entry].message_length  = length;
+		conn->entry++;
+	}
+	else
+	{
+		/* In older protocols output one contents per one line */
+		switch (type)
+		{
+			case BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fputnbytes(conn->Pfdebug, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * put_frontend_entry:
+ * 	After output frontend message length, output organized frontend message.
+ */
+static void put_frontend_entry(PGconn *conn)
+{
+	int 		i;
+	int 		message_addr;
+	int 		length;
+
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	for (i = 0; i < conn->entry; i++)
+	{
+		message_addr = conn->frontend_entry[i].message_addr;
+		length = conn->frontend_entry[i].message_length;
+
+		switch (conn->frontend_entry[i].type)
+		{
+			case BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				message_Byte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case STRING:
+				message_String(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case NCHAR:
+				message_nchar(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				message_Int(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				message_Int(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	conn->entry = 0;
+	initialize_message_log(conn);
+}
+
+/*
+ * getCurrentTime: get current time for trace log output
+ */
+static void
+getCurrentTime(char* currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -99,12 +466,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		message_Byte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -115,11 +481,49 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		message_entry(conn, BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * message_Byte1: output 1 char message to the log
+ */
+static void
+message_Byte1(PGconn *conn, char v, CommunicationDirection direction)
+{
+	char *command_text;
+	char *direction_text = direction == FROM_BACKEND ? "<" : ">";
+	char current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state) {
+			case LOG_COMMAND:
+				command_text = message_get_command_text((unsigned char) v, direction);
+				getCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s %s ", current_time, direction_text, command_text);
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				conn->entry = 0;
+				break;
+
+			case LOG_BODY:
+				fprintf(conn->Pfdebug, "%c ", v);
+				message_state_transaction(sizeof(v), conn);
+				break;
+
+			default:
+				message_invalid_protocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -153,8 +557,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		message_String(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -182,12 +585,39 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		message_entry(conn, STRING, strlen(s) + 1);
 
 	return 0;
 }
 
 /*
+ * message_String: output a a null-terminated string to the log
+ */
+static void
+message_String(PGconn *conn, const char *v, int length, CommunicationDirection direction)
+{
+	if(length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_BODY:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				message_state_transaction(length, conn);
+				break;
+
+			default:
+				message_invalid_protocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
  */
@@ -203,11 +633,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		message_nchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -227,11 +653,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		message_nchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -249,13 +671,39 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
+		message_entry(conn, NCHAR, len);
+
+	return 0;
+}
+
+/*
+ * message_nchar: output a string of exactly len bytes message to the log
+ */
+static void
+message_nchar(PGconn *conn, const char *v, int length, CommunicationDirection direction)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
+		switch (conn->logging_message.state)
+		{
+			case LOG_BODY:
+				fprintf(conn->Pfdebug, "\'");
+				fputnbytes(conn->Pfdebug, v, length);
+				fprintf(conn->Pfdebug, "\' ");
+				message_state_transaction(length, conn);
+				break;
+
+			default:
+				message_invalid_protocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fputnbytes(conn->Pfdebug, v, length);
 		fprintf(conn->Pfdebug, "\n");
 	}
-
-	return 0;
 }
 
 /*
@@ -293,7 +741,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		message_Int(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -315,11 +763,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				message_entry(conn, INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				message_entry(conn, INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -327,11 +779,40 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+/*
+ * message_Int: output a 2 or 4 bytes integer message to the log
+ */
+static void
+message_Int(PGconn *conn, int v, int length, CommunicationDirection direct)
+{
+	char *prefix = length == 4 ? "" : "#";
 
-	return 0;
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				conn->logging_message.state = LOG_BODY;
+				message_state_transaction(0, conn);
+				break;
+
+			case LOG_BODY:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				message_state_transaction(length, conn);
+				break;
+
+			default:
+				message_invalid_protocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
 }
 
 /*
@@ -549,8 +1030,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		message_Byte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -586,15 +1066,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			message_Int(conn, (int) msgLen, 4, FROM_FRONTEND);
+			put_frontend_entry(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index d582cfd..6013c6a 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -125,6 +125,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				message_state_transaction(msgLength, conn);
 			return;
 		}
 
@@ -158,7 +161,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					message_state_transaction(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1221ea9..c5bc9d4 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,44 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/* Logging message */
+typedef enum CommunicationDirection
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} CommunicationDirection;
+
+typedef enum State
+{
+	LOG_COMMAND,
+	LOG_LENGTH,
+	LOG_BODY
+} State;
+
+typedef struct _LoggingMsg
+{
+	State state;
+	int length;
+	char command;
+} LoggingMsg;
+
+/* Organizing frontend message.  */
+typedef enum Type
+{
+	BYTE1,
+	STRING,
+	NCHAR,
+	INT16,
+	INT32
+} Type;
+
+typedef struct _Frontend_Entry
+{
+	Type type;
+	int message_addr;
+	int message_length;
+} Frontend_Entry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -370,6 +408,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* Organize trace info to output one line */
+	LoggingMsg logging_message;
+	Frontend_Entry frontend_entry[MAXPGPATH];
+	int		entry;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -656,6 +699,7 @@ extern int pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 			time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void	message_state_transaction(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#51Kyotaro HORIGUCHI
horiguchi.kyotaro@lab.ntt.co.jp
In reply to: Iwata, Aya (#50)
Re: libpq debug log

Hello. Thank you for the new patch.

At Tue, 9 Apr 2019 06:19:32 +0000, "Iwata, Aya" <iwata.aya@jp.fujitsu.com> wrote in <71E660EB361DF14299875B198D4CE5423DF161BA@g01jpexmbkw25>

Hi,

I update patch to improve PQtrace(); output log message in one line.
Please find my attached patch.

How it changed:

The basic idea being:

- Each line is a whole message.
- The line begins with <<< for a message received and >>> for a message

sent.

- Strings in single quotes are those sent/received as a fixed number of

bytes.

- Strings in double quotes are those sent/received as a string.
- 4-byte integers are printed unadorned.
- 2-byte integers are prefixed by #.
- I guess 1-byte integers would need some other prefix, maybe @ or ##.

New log output examples:
The message sent from frontend is like this;
2019-04-04 02:39:51.488 UTC > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"

The message sent from backend is like this;
2019-04-04 02:39:51.489 UTC < RowDescription 35 #1 "set_config" 0 #0 25 #65535 -1 #0

I had a brief look on this.

+/* protocol message name */
+static char *command_text_b[] = {

Couldn't the name be more descriptive? The comment just above
doesn't seem consistent with the variable. The tables are very
sparse. I think the definition could be in more compact form.

+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_BF_MAX (sizeof(command_text_bf) / sizeof(*command_text_bf))

It seems that at least the trailing 0-elements are not required.

+ * message_get_command_text:
+ * 	Get Protocol message text from byte1 identifier
+ */
+static char *
+message_get_command_text(unsigned char c, CommunicationDirection direction)
..
+message_nchar(PGconn *conn, const char *v, int length, CommunicationDirection direction)

Also the function names are not very descriptive.

+message_Int(PGconn *conn, int v, int length, CommunicationDirection direct)

We are not using names mixing CamelCase and undercored there.

+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		text = command_text_bf[c];
+		if (text)
+			return text;
+	}
+
+	if (direction == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		text = command_text_b[c];
+		if (text)
..
+	if (direction == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)

This code is assuming that elements of command_text_bf is
mutually exclusive with command_text_b or _bf. That is, the first
has an element for 'C', others don't have an element in the same
position. But _bf[C] = "CommandComplete" and _f[C] = "Close". Is
it working correctly?

+typedef enum CommunicationDirection

The type CommunicationDirection is two-member enum which is
equivalent to just a boolean. Is there a reason you define that?

+typedef enum State
+typedef enum Type

The name is too generic.
+typedef struct _LoggingMsg
...
+} LoggingMsg;

Why the tag name is prefixed with an underscore?

+typedef struct _Frontend_Entry

The name doesn't give an idea of its characteristics.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#52Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Kyotaro HORIGUCHI (#51)
1 attachment(s)
RE: libpq debug log

Hi Horiguchi-san,

Thank you for your reviewing.
I updated patch. Please see my attached patch.

+/* protocol message name */
+static char *command_text_b[] = {

Couldn't the name be more descriptive? The comment just above doesn't seem
consistent with the variable. The tables are very sparse. I think the
definition could be in more compact form.

Thank you. I changed the description more clear.

+	/* y */ 0,
+	/* z */ 0
+};
+#define COMMAND_BF_MAX (sizeof(command_text_bf) /
+sizeof(*command_text_bf))

It seems that at least the trailing 0-elements are not required.

Sure. I removed.

+ * message_get_command_text:
+ * 	Get Protocol message text from byte1 identifier
+ */
+static char *
+message_get_command_text(unsigned char c, CommunicationDirection
+direction)
..
+message_nchar(PGconn *conn, const char *v, int length,
+CommunicationDirection direction)

Also the function names are not very descriptive.

Thank you. I fixed function names and added descriptions.

+message_Int(PGconn *conn, int v, int length, CommunicationDirection
+direct)

We are not using names mixing CamelCase and undercored there.

+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		text = command_text_bf[c];
+		if (text)
+			return text;
+	}
+
+	if (direction == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		text = command_text_b[c];
+		if (text)
..
+	if (direction == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)

This code is assuming that elements of command_text_bf is mutually exclusive
with command_text_b or _bf. That is, the first has an element for 'C', others
don't have an element in the same position. But _bf[C] = "CommandComplete"
and _f[C] = "Close". Is it working correctly?

Elements sent from both the backend and the frontend are 'c' and 'd'.
There is no same elements in protocol_message_type_b and _bf.
The same applies to protocol_message_type_f and _bf too. So I think it is working correctly.

+typedef enum CommunicationDirection

The type CommunicationDirection is two-member enum which is equivalent to
just a boolean. Is there a reason you define that?

+typedef enum State
+typedef enum Type

The name is too generic.
+typedef struct _LoggingMsg
...
+} LoggingMsg;

Why the tag name is prefixed with an underscore?

+typedef struct _Frontend_Entry

The name doesn't give an idea of its characteristics.

Thank you. I fixed.

Regards,
Aya Iwata

Attachments:

v3-libpq-PQtrace-output-one-line.patchapplication/octet-stream; name=v3-libpq-PQtrace-output-one-line.patchDownload
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ea4c9d2..06c7eea 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,12 +55,329 @@
 #include "port/pg_bswap.h"
 #include "pg_config_paths.h"
 
+/*
+ * protocol types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ * protocol_message_type_bf[]: messages types sent by both backend and frontend
+ *
+ */
+static char *protocol_message_type_b[] = {
+	/* 0 */ 0,
+	/* 1 */ "ParseComplete",
+	/* 2 */ "BindComplete",
+	/* 3 */ "CloseComplete",
+	/* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ "NotificationResponse",
+	/* B */ 0,
+	/* C */ "CommandComplete",
+	/* D */ "DataRow",
+	/* E */ "ErrorResponse",
+	/* F */ 0,
+	/* G */ "CopyInResponse",
+	/* H */ "CopyOutResponse",
+	/* I */ "EmptyQueryResponse",
+	/* J */ 0,
+	/* K */ "BackendKeyData",
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ "NoticeResponse",
+	/* O */ 0,
+	/* P */ 0,
+	/* Q */ 0,
+	/* R */ "Authentication",
+	/* S */ "ParameterStatus",
+	/* T */ "RowDescription",
+	/* U */ 0,
+	/* V */ "FunctionCallResponse",
+	/* W */ "CopyBothResponse",
+	/* X */ 0,
+	/* Y */ 0,
+	/* Z */ "ReadyForQuery",
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* \x60 - \x6d*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* n */ "NoData",
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ "PortalSuspended",
+	/* t */ "ParameterDescription",
+	/* u */ 0,
+	/* v */ "NegotiateProtocolVersion",
+};
+#define COMMAND_B_MAX (sizeof(protocol_message_type_b) / sizeof(*protocol_message_type_b))
+
+static char *protocol_message_type_f[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ 0,
+	/* B */ "Bind",
+	/* C */ "Close",
+	/* D */ "Describe",
+	/* E */ "Execute",
+	/* F */ "FunctionCall",
+	/* G */ 0,
+	/* H */ "Flush",
+	/* I - O       */ 0, 0, 0, 0, 0, 0, 0,
+	/* P */ "Parse",
+	/* Q */ "Query",
+	/* R */ 0,
+	/* S */ "Sync",
+	/* T - W       */ 0, 0, 0, 0,
+	/* X */ "Terminate",
+	/* Y */ 0,
+	/* Z */ 0,
+	/* \x5b - \x5f */ 0, 0, 0, 0, 0,
+	/* ` - e       */ 0, 0, 0, 0, 0, 0,
+	/* f */ "CopyFail",
+	/* g - o       */ 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* p */ "AuthnenticationResponse",
+};
+#define COMMAND_F_MAX (sizeof(protocol_message_type_f) / sizeof(*protocol_message_type_f))
+
+static char *protocol_message_type_bf[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x40 - \x4f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x50 - \x5f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ "CopyDone",
+	/* d */ "CopyData",
+};
+#define COMMAND_BF_MAX (sizeof(protocol_message_type_bf) / sizeof(*protocol_message_type_bf))
+
 
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void	fputnbytes(FILE *f, const char *str, size_t n);
+static void 	pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void 	pqLogMsgString(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgnchar(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void 	pqGetCurrentTime(char* currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * pqGetProtocolMsgType:
+ * 	Get a protocol type from first byte identifier
+ */
+static char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	char *message_type = 0;
+
+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		message_type = protocol_message_type_bf[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		message_type = protocol_message_type_b[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)
+	{
+		message_type = protocol_message_type_f[c];
+		if (message_type)
+			return message_type;
+	}
+
+	return "UnknownCommand";
+}
+
+/* pqInitMsgLog: Initializing logging message */
+static void
+pqInitMsgLog(PGconn *conn) {
+	conn->logging_message.state = LOG_FIRST_BYTE;
+	conn->logging_message.length = 0;
+}
+
+/* pqLogInvalidProtocol: Output a message the protocol is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqLogLineBreak:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+pqLogLineBreak(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqInitMsgLog(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg: Store message addresses that frontend sends.
+ *
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void pqStoreFrontendMsg(PGconn *conn ,PGLogMsgDataType type, int length)
+{
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->frontend_entry[conn->nMsgEntrys].type  = type;
+		conn->frontend_entry[conn->nMsgEntrys].message_addr  = conn->outMsgEnd - length;
+		conn->frontend_entry[conn->nMsgEntrys].message_length  = length;
+		conn->nMsgEntrys++;
+	}
+	else
+	{
+		/* Output one content per one line in older protocol version */
+		switch (type)
+		{
+			case BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fputnbytes(conn->Pfdebug, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg:
+ * 	Output frontend message contents after the message length.
+ */
+static void pqLogFrontendMsg(PGconn *conn)
+{
+	int 		i;
+	int 		message_addr;
+	int 		length;
+
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	for (i = 0; i < conn->nMsgEntrys; i++)
+	{
+		message_addr = conn->frontend_entry[i].message_addr;
+		length = conn->frontend_entry[i].message_length;
+
+		switch (conn->frontend_entry[i].type)
+		{
+			case BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	conn->nMsgEntrys = 0;
+	pqInitMsgLog(conn);
+}
+
+/*
+ * pqGetCurrentTime: get current time for trace log output
+ */
+static void
+pqGetCurrentTime(char* currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -99,12 +418,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -115,11 +433,51 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char *protocol_message_type;
+	char *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	char current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_FIRST_BYTE:
+				protocol_message_type = pqGetProtocolMsgType((unsigned char) v, commsource);
+				pqGetCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s %s ", current_time, message_source, protocol_message_type);
+				/* Change the state to get protocol message length */
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				conn->nMsgEntrys = 0;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqLogLineBreak(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -153,8 +511,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -182,12 +539,39 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, STRING, strlen(s) + 1);
 
 	return 0;
 }
 
 /*
+ * pqLogMsgString: output a a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if(length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
  */
@@ -203,11 +587,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -227,11 +607,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -249,13 +625,39 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
+		pqStoreFrontendMsg(conn, NCHAR, len);
+
+	return 0;
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fputnbytes(conn->Pfdebug, v, length);
+				fprintf(conn->Pfdebug, "\' ");
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fputnbytes(conn->Pfdebug, v, length);
 		fprintf(conn->Pfdebug, "\n");
 	}
-
-	return 0;
 }
 
 /*
@@ -293,7 +695,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -315,11 +717,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -327,11 +733,41 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char *prefix = length == 4 ? "" : "#";
 
-	return 0;
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				/* Change the state to log protocol message contents */
+				conn->logging_message.state = LOG_CONTENTS;
+				pqLogLineBreak(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
 }
 
 /*
@@ -549,8 +985,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -586,15 +1021,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index d582cfd..113892f 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -125,6 +125,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				pqLogLineBreak(msgLength, conn);
 			return;
 		}
 
@@ -158,7 +161,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					pqLogLineBreak(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1221ea9..baf76f6 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,51 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* PGLogState defines the state of the Logging message state machine */
+typedef enum
+{
+	LOG_FIRST_BYTE,	/* logging the first byte identifing the protocol message type */
+	LOG_LENGTH,	/* logging protocol message length */
+	LOG_CONTENTS	/* logging protocol message contents */
+} PGLogState;
+
+/* Protocol message */
+typedef struct PGLogMsg
+{
+	PGLogState state;	/* state of logging message state machine */
+	int length;	/* protocol message length */
+	char command;	/* first one byte of protocol message */
+} PGLogMsg;
+
+/* Frontend message data type */
+typedef enum
+{
+	BYTE1,
+	STRING,
+	NCHAR,
+	INT16,
+	INT32
+} PGLogMsgDataType;
+
+/* Store frontend message address */
+typedef struct PGFrontendLogMsgEntry
+{
+	PGLogMsgDataType type;
+	int message_addr;
+	int message_length;
+} PGFrontendLogMsgEntry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -370,6 +415,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* Trace log info to output one line */
+	PGLogMsg logging_message;
+	PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
+	int		nMsgEntrys;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -656,6 +706,7 @@ extern int pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 			time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void	pqLogLineBreak(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#53Jamison, Kirk
k.jamison@jp.fujitsu.com
In reply to: Iwata, Aya (#52)
RE: libpq debug log

Hi Aya-san,

I tested your v3 patch and it seemed to work on my Linux environment.
However, the CF Patch Tester detected a build failure (probably on Windows).
Refer to: http://commitfest.cputube.org/

Docs:
It would be better to have reference to the documentations of
Frontend/Backend Protocol's "Message Format".

Code:
There are some protocol message types from frontend
that you missed indicating (non BYTE1 types):
CancelRequest (F), StartupMessage (F), SSLRequest (F).

Although I haven't tested those actual protocols,
I assume it will be printed as the following since the length and
message will still be recognized.
ex. Timestamp 8 80877103

So you need to indicate these protocol message types as intended.
ex. Timestamp > SSLRequest 8 80877103

Regards,
Kirk Jamison

#54Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Jamison, Kirk (#53)
1 attachment(s)
RE: libpq debug log

Hi Kirk,

Thank you for your reviewing.

Docs:
It would be better to have reference to the documentations of Frontend/Backend
Protocol's "Message Format".

I added a link to "Protocol's Message Formats" and little explanation to PQtrace() documentation.

Code:
There are some protocol message types from frontend that you missed indicating
(non BYTE1 types):
CancelRequest (F), StartupMessage (F), SSLRequest (F).

Although I haven't tested those actual protocols, I assume it will be printed
as the following since the length and message will still be recognized.
ex. Timestamp 8 80877103

So you need to indicate these protocol message types as intended.
ex. Timestamp > SSLRequest 8 80877103

Thank you. I changed code to output these information.
For that, I added code to check the int32 content which StartupMessage (F) and SSLRequest (F) have.
And CancelRequest is not targeted because it calls send() directly.

Regards,
Aya Iwata

Attachments:

v4-libpq-PQtrace-output-one-line.patchapplication/octet-stream; name=v4-libpq-PQtrace-output-one-line.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 7f01fcc..b5559ec 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6117,6 +6117,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ea4c9d2..b4e7a5c 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,12 +55,327 @@
 #include "port/pg_bswap.h"
 #include "pg_config_paths.h"
 
+/*
+ * protocol types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ * protocol_message_type_bf[]: messages types sent by both backend and frontend
+ *
+ */
+static char *protocol_message_type_b[] = {
+	/* 0 */ 0,
+	/* 1 */ "ParseComplete",
+	/* 2 */ "BindComplete",
+	/* 3 */ "CloseComplete",
+	/* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ "NotificationResponse",
+	/* B */ 0,
+	/* C */ "CommandComplete",
+	/* D */ "DataRow",
+	/* E */ "ErrorResponse",
+	/* F */ 0,
+	/* G */ "CopyInResponse",
+	/* H */ "CopyOutResponse",
+	/* I */ "EmptyQueryResponse",
+	/* J */ 0,
+	/* K */ "BackendKeyData",
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ "NoticeResponse",
+	/* O */ 0,
+	/* P */ 0,
+	/* Q */ 0,
+	/* R */ "Authentication",
+	/* S */ "ParameterStatus",
+	/* T */ "RowDescription",
+	/* U */ 0,
+	/* V */ "FunctionCallResponse",
+	/* W */ "CopyBothResponse",
+	/* X */ 0,
+	/* Y */ 0,
+	/* Z */ "ReadyForQuery",
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* \x60 - \x6d*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* n */ "NoData",
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ "PortalSuspended",
+	/* t */ "ParameterDescription",
+	/* u */ 0,
+	/* v */ "NegotiateProtocolVersion",
+};
+#define COMMAND_B_MAX (sizeof(protocol_message_type_b) / sizeof(*protocol_message_type_b))
+
+static char *protocol_message_type_f[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ 0,
+	/* B */ "Bind",
+	/* C */ "Close",
+	/* D */ "Describe",
+	/* E */ "Execute",
+	/* F */ "FunctionCall",
+	/* G */ 0,
+	/* H */ "Flush",
+	/* I - O       */ 0, 0, 0, 0, 0, 0, 0,
+	/* P */ "Parse",
+	/* Q */ "Query",
+	/* R */ 0,
+	/* S */ "Sync",
+	/* T - W       */ 0, 0, 0, 0,
+	/* X */ "Terminate",
+	/* Y */ 0,
+	/* Z */ 0,
+	/* \x5b - \x65 */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* f */ "CopyFail",
+	/* g - o       */ 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* p */ "AuthnenticationResponse",
+};
+#define COMMAND_F_MAX (sizeof(protocol_message_type_f) / sizeof(*protocol_message_type_f))
+
+static char *protocol_message_type_bf[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x40 - \x4f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x50 - \x5f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ "CopyDone",
+	/* d */ "CopyData",
+};
+#define COMMAND_BF_MAX (sizeof(protocol_message_type_bf) / sizeof(*protocol_message_type_bf))
+
 
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 			  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void	fputnbytes(FILE *f, const char *str, size_t n);
+static void 	pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void 	pqLogMsgString(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgnchar(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void 	pqGetCurrentTime(char* currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * pqGetProtocolMsgType:
+ * 	Get a protocol type from first byte identifier
+ */
+static char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	char *message_type = 0;
+
+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		message_type = protocol_message_type_bf[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		message_type = protocol_message_type_b[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)
+	{
+		message_type = protocol_message_type_f[c];
+		if (message_type)
+			return message_type;
+	}
+
+	return "UnknownCommand";
+}
+
+/* pqInitMsgLog: Initializing logging message */
+static void
+pqInitMsgLog(PGconn *conn) {
+	conn->logging_message.state = LOG_FIRST_BYTE;
+	conn->logging_message.length = 0;
+}
+
+/* pqLogInvalidProtocol: Output a message the protocol is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqLogLineBreak:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+pqLogLineBreak(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqInitMsgLog(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg: Store message addresses that frontend sends.
+ *
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void pqStoreFrontendMsg(PGconn *conn ,PGLogMsgDataType type, int length)
+{
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->frontend_entry[conn->nMsgEntrys].type  = type;
+		conn->frontend_entry[conn->nMsgEntrys].message_addr  = conn->outMsgEnd - length;
+		conn->frontend_entry[conn->nMsgEntrys].message_length  = length;
+		conn->nMsgEntrys++;
+	}
+	else
+	{
+		/* Output one content per one line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fputnbytes(conn->Pfdebug, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg:
+ * 	Output frontend message contents after the message length.
+ */
+static void pqLogFrontendMsg(PGconn *conn)
+{
+	int 		i;
+	int 		message_addr;
+	int 		length;
+
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	for (i = 0; i < conn->nMsgEntrys; i++)
+	{
+		message_addr = conn->frontend_entry[i].message_addr;
+		length = conn->frontend_entry[i].message_length;
+
+		switch (conn->frontend_entry[i].type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	pqInitMsgLog(conn);
+}
+
+/*
+ * pqGetCurrentTime: get current time for trace log output
+ */
+static void
+pqGetCurrentTime(char* currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -99,12 +416,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -115,11 +431,56 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char *protocol_message_type;
+	char *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	char current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_FIRST_BYTE:
+				pqGetCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s " , current_time, message_source);
+				/* If there is no first 1 byte protocol message, */
+				if (v == ' ')
+					return;
+
+				protocol_message_type = pqGetProtocolMsgType((unsigned char) v, commsource);
+				fprintf(conn->Pfdebug, "%s ", protocol_message_type);
+				/* Change the state to get protocol message length */
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				conn->nMsgEntrys = 0;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqLogLineBreak(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -153,8 +514,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -182,12 +542,39 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
 
 /*
+ * pqLogMsgString: output a a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if(length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
  */
@@ -203,11 +590,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -227,11 +610,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -249,13 +628,39 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
+
+	return 0;
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fputnbytes(conn->Pfdebug, v, length);
+				fprintf(conn->Pfdebug, "\' ");
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fputnbytes(conn->Pfdebug, v, length);
 		fprintf(conn->Pfdebug, "\n");
 	}
-
-	return 0;
 }
 
 /*
@@ -293,7 +698,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -315,11 +720,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -327,11 +736,64 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char *prefix = length == 4 ? "" : "#";
+	char *message_type = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+	int 		message_addr;
 
-	return 0;
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			/* Output message type here
+			 * for protocol messages that do not have the first byte. */
+			case LOG_FIRST_BYTE:
+				if (conn->nMsgEntrys > 0)
+				{
+					message_addr = conn->frontend_entry[0].message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->logging_message.state = LOG_LENGTH;
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				/* Change the state to log protocol message contents */
+				conn->logging_message.state = LOG_CONTENTS;
+				pqLogLineBreak(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
 }
 
 /*
@@ -549,8 +1011,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -586,15 +1047,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index d582cfd..113892f 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -125,6 +125,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				pqLogLineBreak(msgLength, conn);
 			return;
 		}
 
@@ -158,7 +161,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					pqLogLineBreak(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1221ea9..9e7bd39 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,51 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* PGLogState defines the state of the Logging message state machine */
+typedef enum
+{
+	LOG_FIRST_BYTE,	/* logging the first byte identifing the protocol message type */
+	LOG_LENGTH,	/* logging protocol message length */
+	LOG_CONTENTS	/* logging protocol message contents */
+} PGLogState;
+
+/* Protocol message */
+typedef struct PGLogMsg
+{
+	PGLogState state;	/* state of logging message state machine */
+	int length;	/* protocol message length */
+	char command;	/* first one byte of protocol message */
+} PGLogMsg;
+
+/* Frontend message data type */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+/* Store frontend message address */
+typedef struct PGFrontendLogMsgEntry
+{
+	PGLogMsgDataType type;
+	int message_addr;
+	int message_length;
+} PGFrontendLogMsgEntry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -370,6 +415,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* Trace log info to output one line */
+	PGLogMsg logging_message;
+	PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
+	int		nMsgEntrys;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -656,6 +706,7 @@ extern int pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 			time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void	pqLogLineBreak(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#55Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#54)
1 attachment(s)
RE: libpq debug log

Hi,

I rebased my patch from head. Please find my attached patch.

Regard,
Aya Iwata

Attachments:

v5-libpq-PQtrace-output-one-line.patchapplication/octet-stream; name=v5-libpq-PQtrace-output-one-line.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 7f01fcc..b5559ec 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6117,6 +6117,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2d44845..f66bbca 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,12 +55,327 @@
 #include "port/pg_bswap.h"
 #include "pg_config_paths.h"
 
+/*
+ * protocol types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ * protocol_message_type_bf[]: messages types sent by both backend and frontend
+ *
+ */
+static char *protocol_message_type_b[] = {
+	/* 0 */ 0,
+	/* 1 */ "ParseComplete",
+	/* 2 */ "BindComplete",
+	/* 3 */ "CloseComplete",
+	/* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ "NotificationResponse",
+	/* B */ 0,
+	/* C */ "CommandComplete",
+	/* D */ "DataRow",
+	/* E */ "ErrorResponse",
+	/* F */ 0,
+	/* G */ "CopyInResponse",
+	/* H */ "CopyOutResponse",
+	/* I */ "EmptyQueryResponse",
+	/* J */ 0,
+	/* K */ "BackendKeyData",
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ "NoticeResponse",
+	/* O */ 0,
+	/* P */ 0,
+	/* Q */ 0,
+	/* R */ "Authentication",
+	/* S */ "ParameterStatus",
+	/* T */ "RowDescription",
+	/* U */ 0,
+	/* V */ "FunctionCallResponse",
+	/* W */ "CopyBothResponse",
+	/* X */ 0,
+	/* Y */ 0,
+	/* Z */ "ReadyForQuery",
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* \x60 - \x6d*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* n */ "NoData",
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ "PortalSuspended",
+	/* t */ "ParameterDescription",
+	/* u */ 0,
+	/* v */ "NegotiateProtocolVersion",
+};
+#define COMMAND_B_MAX (sizeof(protocol_message_type_b) / sizeof(*protocol_message_type_b))
+
+static char *protocol_message_type_f[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ 0,
+	/* B */ "Bind",
+	/* C */ "Close",
+	/* D */ "Describe",
+	/* E */ "Execute",
+	/* F */ "FunctionCall",
+	/* G */ 0,
+	/* H */ "Flush",
+	/* I - O       */ 0, 0, 0, 0, 0, 0, 0,
+	/* P */ "Parse",
+	/* Q */ "Query",
+	/* R */ 0,
+	/* S */ "Sync",
+	/* T - W       */ 0, 0, 0, 0,
+	/* X */ "Terminate",
+	/* Y */ 0,
+	/* Z */ 0,
+	/* \x5b - \x65 */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* f */ "CopyFail",
+	/* g - o       */ 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* p */ "AuthnenticationResponse",
+};
+#define COMMAND_F_MAX (sizeof(protocol_message_type_f) / sizeof(*protocol_message_type_f))
+
+static char *protocol_message_type_bf[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x40 - \x4f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x50 - \x5f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ "CopyDone",
+	/* d */ "CopyData",
+};
+#define COMMAND_BF_MAX (sizeof(protocol_message_type_bf) / sizeof(*protocol_message_type_bf))
+
 
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void	fputnbytes(FILE *f, const char *str, size_t n);
+static void 	pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void 	pqLogMsgString(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgnchar(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void 	pqGetCurrentTime(char* currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * pqGetProtocolMsgType:
+ * 	Get a protocol type from first byte identifier
+ */
+static char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	char *message_type = 0;
+
+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		message_type = protocol_message_type_bf[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		message_type = protocol_message_type_b[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)
+	{
+		message_type = protocol_message_type_f[c];
+		if (message_type)
+			return message_type;
+	}
+
+	return "UnknownCommand";
+}
+
+/* pqInitMsgLog: Initializing logging message */
+static void
+pqInitMsgLog(PGconn *conn) {
+	conn->logging_message.state = LOG_FIRST_BYTE;
+	conn->logging_message.length = 0;
+}
+
+/* pqLogInvalidProtocol: Output a message the protocol is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqLogLineBreak:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+pqLogLineBreak(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqInitMsgLog(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg: Store message addresses that frontend sends.
+ *
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void pqStoreFrontendMsg(PGconn *conn ,PGLogMsgDataType type, int length)
+{
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->frontend_entry[conn->nMsgEntrys].type  = type;
+		conn->frontend_entry[conn->nMsgEntrys].message_addr  = conn->outMsgEnd - length;
+		conn->frontend_entry[conn->nMsgEntrys].message_length  = length;
+		conn->nMsgEntrys++;
+	}
+	else
+	{
+		/* Output one content per one line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fputnbytes(conn->Pfdebug, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg:
+ * 	Output frontend message contents after the message length.
+ */
+static void pqLogFrontendMsg(PGconn *conn)
+{
+	int 		i;
+	int 		message_addr;
+	int 		length;
+
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	for (i = 0; i < conn->nMsgEntrys; i++)
+	{
+		message_addr = conn->frontend_entry[i].message_addr;
+		length = conn->frontend_entry[i].message_length;
+
+		switch (conn->frontend_entry[i].type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	pqInitMsgLog(conn);
+}
+
+/*
+ * pqGetCurrentTime: get current time for trace log output
+ */
+static void
+pqGetCurrentTime(char* currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -99,12 +416,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -115,11 +431,56 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char *protocol_message_type;
+	char *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	char current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_FIRST_BYTE:
+				pqGetCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s " , current_time, message_source);
+				/* If there is no first 1 byte protocol message, */
+				if (v == ' ')
+					return;
+
+				protocol_message_type = pqGetProtocolMsgType((unsigned char) v, commsource);
+				fprintf(conn->Pfdebug, "%s ", protocol_message_type);
+				/* Change the state to get protocol message length */
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				conn->nMsgEntrys = 0;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqLogLineBreak(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -153,8 +514,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -182,12 +542,39 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
 
 /*
+ * pqLogMsgString: output a a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if(length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
  */
@@ -203,11 +590,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -227,11 +610,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -249,13 +628,39 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
+
+	return 0;
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fputnbytes(conn->Pfdebug, v, length);
+				fprintf(conn->Pfdebug, "\' ");
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fputnbytes(conn->Pfdebug, v, length);
 		fprintf(conn->Pfdebug, "\n");
 	}
-
-	return 0;
 }
 
 /*
@@ -293,7 +698,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -315,11 +720,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -327,11 +736,64 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char *prefix = length == 4 ? "" : "#";
+	char *message_type = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+	int 		message_addr;
 
-	return 0;
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			/* Output message type here
+			 * for protocol messages that do not have the first byte. */
+			case LOG_FIRST_BYTE:
+				if (conn->nMsgEntrys > 0)
+				{
+					message_addr = conn->frontend_entry[0].message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->logging_message.state = LOG_LENGTH;
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				/* Change the state to log protocol message contents */
+				conn->logging_message.state = LOG_CONTENTS;
+				pqLogLineBreak(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
 }
 
 /*
@@ -549,8 +1011,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -586,15 +1047,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 467563d..0ecb32f 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -125,6 +125,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				pqLogLineBreak(msgLength, conn);
 			return;
 		}
 
@@ -158,7 +161,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					pqLogLineBreak(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c0b8e3f..59d0f98 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,51 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* PGLogState defines the state of the Logging message state machine */
+typedef enum
+{
+	LOG_FIRST_BYTE,	/* logging the first byte identifing the protocol message type */
+	LOG_LENGTH,	/* logging protocol message length */
+	LOG_CONTENTS	/* logging protocol message contents */
+} PGLogState;
+
+/* Protocol message */
+typedef struct PGLogMsg
+{
+	PGLogState state;	/* state of logging message state machine */
+	int length;	/* protocol message length */
+	char command;	/* first one byte of protocol message */
+} PGLogMsg;
+
+/* Frontend message data type */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+/* Store frontend message address */
+typedef struct PGFrontendLogMsgEntry
+{
+	PGLogMsgDataType type;
+	int message_addr;
+	int message_length;
+} PGFrontendLogMsgEntry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -370,6 +415,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* Trace log info to output one line */
+	PGLogMsg logging_message;
+	PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
+	int		nMsgEntrys;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -656,6 +706,7 @@ extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 						time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void	pqLogLineBreak(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#56Iwata, Aya
iwata.aya@jp.fujitsu.com
In reply to: Iwata, Aya (#55)
1 attachment(s)
RE: libpq debug log

Hi,

This is a summary of the whole thread.
I am currently improving PQtrace() by adjusting its output to one line per protocol message as per the advice of reviewers.

Purpose:
If a problem occurs, such as a slow query, you want to know which query takes time.
In Current libpq, there is PQtrace(FILE *filename) facility to output exchanging protocol messages to file.
But I think current PQtrace() has following issues:
* The output is confusing. It is difficult to analyze information because it is printed one by one, and only a character representing a message (ex. printed 'T' means RowDescription).
* Timestamp is not output. So we cannot identify which process took a long time. That would be possible when we compare timestamps.
* PQtrace() code must be included in libpq application's source code. If you want to get log, you should change code and re-compile it for logging. Some application cannot do this.

Compared to tcpdump:
There is tcpdump for similar use, but it has the following problems:
- Windows users cannot use it.
- If the communication is encrypted, it is possible that you may not see the information you want as explained by Andres.
- Information can only be retrieved by limited users due to OS permissions.

Solution:
Work on following improvements in order:
1. Adjusting it to emit one line per protocol message and output timestamp.
2. Enables logging control without recompiling the application.
I thought it would be better to control it with parameters. However since this method is controversial (Security implications, etc.), we will consider a good method after completing 1.

Latest patch just contains 1. Hence, the usage of this feature is the same as current PQtrace().

Example of log output:
In current PQtrace log:

To backend> Msg Q
To backend> "SELECT pg_catalog.set_config('search_path', '', false)"
To backend> Msg complete, length 60

I changed like this:

2019-04-04 02:39:51.488 UTC > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"

I appreciate your advice regarding the one line protocol message. Thank you in advance.

Regards,
Aya Iwata

Attachments:

v6-libpq-PQtrace-output-one-line.patchapplication/octet-stream; name=v6-libpq-PQtrace-output-one-line.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 7f01fcc..b5559ec 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6117,6 +6117,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2d44845..f66bbca 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,12 +55,327 @@
 #include "port/pg_bswap.h"
 #include "pg_config_paths.h"
 
+/*
+ * protocol types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ * protocol_message_type_bf[]: messages types sent by both backend and frontend
+ *
+ */
+static char *protocol_message_type_b[] = {
+	/* 0 */ 0,
+	/* 1 */ "ParseComplete",
+	/* 2 */ "BindComplete",
+	/* 3 */ "CloseComplete",
+	/* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ "NotificationResponse",
+	/* B */ 0,
+	/* C */ "CommandComplete",
+	/* D */ "DataRow",
+	/* E */ "ErrorResponse",
+	/* F */ 0,
+	/* G */ "CopyInResponse",
+	/* H */ "CopyOutResponse",
+	/* I */ "EmptyQueryResponse",
+	/* J */ 0,
+	/* K */ "BackendKeyData",
+	/* L */ 0,
+	/* M */ 0,
+	/* N */ "NoticeResponse",
+	/* O */ 0,
+	/* P */ 0,
+	/* Q */ 0,
+	/* R */ "Authentication",
+	/* S */ "ParameterStatus",
+	/* T */ "RowDescription",
+	/* U */ 0,
+	/* V */ "FunctionCallResponse",
+	/* W */ "CopyBothResponse",
+	/* X */ 0,
+	/* Y */ 0,
+	/* Z */ "ReadyForQuery",
+	/* \x5b - \x5f*/ 0, 0, 0, 0, 0,
+	/* \x60 - \x6d*/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* n */ "NoData",
+	/* o */ 0,
+	/* p */ 0,
+	/* q */ 0,
+	/* r */ 0,
+	/* s */ "PortalSuspended",
+	/* t */ "ParameterDescription",
+	/* u */ 0,
+	/* v */ "NegotiateProtocolVersion",
+};
+#define COMMAND_B_MAX (sizeof(protocol_message_type_b) / sizeof(*protocol_message_type_b))
+
+static char *protocol_message_type_f[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* @ */ 0,
+	/* A */ 0,
+	/* B */ "Bind",
+	/* C */ "Close",
+	/* D */ "Describe",
+	/* E */ "Execute",
+	/* F */ "FunctionCall",
+	/* G */ 0,
+	/* H */ "Flush",
+	/* I - O       */ 0, 0, 0, 0, 0, 0, 0,
+	/* P */ "Parse",
+	/* Q */ "Query",
+	/* R */ 0,
+	/* S */ "Sync",
+	/* T - W       */ 0, 0, 0, 0,
+	/* X */ "Terminate",
+	/* Y */ 0,
+	/* Z */ 0,
+	/* \x5b - \x65 */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* f */ "CopyFail",
+	/* g - o       */ 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* p */ "AuthnenticationResponse",
+};
+#define COMMAND_F_MAX (sizeof(protocol_message_type_f) / sizeof(*protocol_message_type_f))
+
+static char *protocol_message_type_bf[] = {
+	/* \0   - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x40 - \x4f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* \x50 - \x5f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	/* ` */ 0,
+	/* a */ 0,
+	/* b */ 0,
+	/* c */ "CopyDone",
+	/* d */ "CopyData",
+};
+#define COMMAND_BF_MAX (sizeof(protocol_message_type_bf) / sizeof(*protocol_message_type_bf))
+
 
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void	fputnbytes(FILE *f, const char *str, size_t n);
+static void 	pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void 	pqLogMsgString(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgnchar(PGconn *conn, const char *v, int length,
+				PGCommSource commsource);
+static void 	pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void 	pqGetCurrentTime(char* currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * pqGetProtocolMsgType:
+ * 	Get a protocol type from first byte identifier
+ */
+static char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	char *message_type = 0;
+
+	if (c >= 0 && c < COMMAND_BF_MAX)
+	{
+		message_type = protocol_message_type_bf[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_BACKEND && c >= 0 && c < COMMAND_B_MAX)
+	{
+		message_type = protocol_message_type_b[c];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_FRONTEND && c >= 0 && c < COMMAND_F_MAX)
+	{
+		message_type = protocol_message_type_f[c];
+		if (message_type)
+			return message_type;
+	}
+
+	return "UnknownCommand";
+}
+
+/* pqInitMsgLog: Initializing logging message */
+static void
+pqInitMsgLog(PGconn *conn) {
+	conn->logging_message.state = LOG_FIRST_BYTE;
+	conn->logging_message.length = 0;
+}
+
+/* pqLogInvalidProtocol: Output a message the protocol is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqLogLineBreak:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+pqLogLineBreak(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqInitMsgLog(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg: Store message addresses that frontend sends.
+ *
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void pqStoreFrontendMsg(PGconn *conn ,PGLogMsgDataType type, int length)
+{
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->frontend_entry[conn->nMsgEntrys].type  = type;
+		conn->frontend_entry[conn->nMsgEntrys].message_addr  = conn->outMsgEnd - length;
+		conn->frontend_entry[conn->nMsgEntrys].message_length  = length;
+		conn->nMsgEntrys++;
+	}
+	else
+	{
+		/* Output one content per one line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fputnbytes(conn->Pfdebug, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg:
+ * 	Output frontend message contents after the message length.
+ */
+static void pqLogFrontendMsg(PGconn *conn)
+{
+	int 		i;
+	int 		message_addr;
+	int 		length;
+
+	char 		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+
+	for (i = 0; i < conn->nMsgEntrys; i++)
+	{
+		message_addr = conn->frontend_entry[i].message_addr;
+		length = conn->frontend_entry[i].message_length;
+
+		switch (conn->frontend_entry[i].type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+								length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	pqInitMsgLog(conn);
+}
+
+/*
+ * pqGetCurrentTime: get current time for trace log output
+ */
+static void
+pqGetCurrentTime(char* currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+				localTime.wYear, localTime.wMonth, localTime.wDay,
+				localTime.wHour, localTime.wMinute, localTime.wSecond,
+				localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm *tm;
+	char timezone[100];
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			1900+ tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -99,12 +416,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -115,11 +431,56 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char *protocol_message_type;
+	char *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	char current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_FIRST_BYTE:
+				pqGetCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s " , current_time, message_source);
+				/* If there is no first 1 byte protocol message, */
+				if (v == ' ')
+					return;
+
+				protocol_message_type = pqGetProtocolMsgType((unsigned char) v, commsource);
+				fprintf(conn->Pfdebug, "%s ", protocol_message_type);
+				/* Change the state to get protocol message length */
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				conn->nMsgEntrys = 0;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqLogLineBreak(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -153,8 +514,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -182,12 +542,39 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
 
 /*
+ * pqLogMsgString: output a a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if(length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
  */
@@ -203,11 +590,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -227,11 +610,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -249,13 +628,39 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
+
+	return 0;
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fputnbytes(conn->Pfdebug, v, length);
+				fprintf(conn->Pfdebug, "\' ");
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fputnbytes(conn->Pfdebug, v, length);
 		fprintf(conn->Pfdebug, "\n");
 	}
-
-	return 0;
 }
 
 /*
@@ -293,7 +698,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -315,11 +720,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -327,11 +736,64 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char *prefix = length == 4 ? "" : "#";
+	char *message_type = 0;
+	uint32		result32 = 0;
+	int		result = 0;
+	int 		message_addr;
 
-	return 0;
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			/* Output message type here
+			 * for protocol messages that do not have the first byte. */
+			case LOG_FIRST_BYTE:
+				if (conn->nMsgEntrys > 0)
+				{
+					message_addr = conn->frontend_entry[0].message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->logging_message.state = LOG_LENGTH;
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				/* Change the state to log protocol message contents */
+				conn->logging_message.state = LOG_CONTENTS;
+				pqLogLineBreak(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
 }
 
 /*
@@ -549,8 +1011,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -586,15 +1047,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index bbba48b..3769a58 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -125,6 +125,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				pqLogLineBreak(msgLength, conn);
 			return;
 		}
 
@@ -158,7 +161,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					pqLogLineBreak(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index fcf2bc2..9700429 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,51 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* PGLogState defines the state of the Logging message state machine */
+typedef enum
+{
+	LOG_FIRST_BYTE,	/* logging the first byte identifing the protocol message type */
+	LOG_LENGTH,	/* logging protocol message length */
+	LOG_CONTENTS	/* logging protocol message contents */
+} PGLogState;
+
+/* Protocol message */
+typedef struct PGLogMsg
+{
+	PGLogState state;	/* state of logging message state machine */
+	int length;	/* protocol message length */
+	char command;	/* first one byte of protocol message */
+} PGLogMsg;
+
+/* Frontend message data type */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+/* Store frontend message address */
+typedef struct PGFrontendLogMsgEntry
+{
+	PGLogMsgDataType type;
+	int message_addr;
+	int message_length;
+} PGFrontendLogMsgEntry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -370,6 +415,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* Trace log info to output one line */
+	PGLogMsg logging_message;
+	PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
+	int		nMsgEntrys;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -656,6 +706,7 @@ extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 						time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void	pqLogLineBreak(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#57Alvaro Herrera from 2ndQuadrant
alvherre@alvh.no-ip.org
In reply to: Iwata, Aya (#56)
Re: libpq debug log

Hello,

Nice patch. I think there's pretty near consensus that this is
something we want, and that the output format of one trace msg per libpq
msg is roughly okay. (I'm not sure there's 100% agreement on this last
point, but it seems close enough.)

I changed like this:

2019-04-04 02:39:51.488 UTC > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"

The "59" there seems quite odd, though.

* What is this in pg_conn? I don't understand why this array is of size
MAXPGPATH.
+ PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
This seems to make pg_conn much larger than we'd like.

Minor review points:

* Do we need COMMAND_B_MIN etc to reduce the number of zeroes at the
beginning of the tables?

* The place where you define TRACELOG_TIME_SIZE seems like it'll be
unavoidably out of date if somebody changes the format string. I'd put
that #define next to where the fmt appears.

* "output a a null-terminated" has duplicated "a"

* We don't add braces around single-statement blocks (pqPutMsgEnd)

* Please pgindent.

Thanks

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#58iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Alvaro Herrera from 2ndQuadrant (#57)
1 attachment(s)
RE: libpq debug log

Hello,

Thank you for your review.
I update patch. Please find attached my patch.

2019-04-04 02:39:51.488 UTC > Query 59 "SELECT

pg_catalog.set_config('search_path', '', false)"

The "59" there seems quite odd, though.

Could you explain more detail about this?

"59" is length of protocol message contents. (It does not contain first 1 byte.)
This order is based on the message format.
https://www.postgresql.org/docs/current/protocol-message-formats.html

* What is this in pg_conn? I don't understand why this array is of size
MAXPGPATH.
+ PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
This seems to make pg_conn much larger than we'd like.

Sure. I remove this and change code to use list.

Minor review points:

I accept all these review points.

Regards,
Aya Iwata

Attachments:

v7-libpq-PQtrace-output-one-line.patchapplication/octet-stream; name=v7-libpq-PQtrace-output-one-line.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 258b09c..f62b014 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5714,6 +5714,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index a7c08c5..5747ffa 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,11 +55,351 @@
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
 
+/*
+ * protocol types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ * protocol_message_type_bf[]: messages types sent by both backend and frontend
+ *
+ */
+static char *protocol_message_type_b[] = {
+	 /* 1 */ "ParseComplete",
+	 /* 2 */ "BindComplete",
+	 /* 3 */ "CloseComplete",
+	 /* \x04 - \x0f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* \x10 - \x1f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* \x20 - \x2f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* \x30 - \x3f */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* @ */ 0,
+	 /* A */ "NotificationResponse",
+	 /* B */ 0,
+	 /* C */ "CommandComplete",
+	 /* D */ "DataRow",
+	 /* E */ "ErrorResponse",
+	 /* F */ 0,
+	 /* G */ "CopyInResponse",
+	 /* H */ "CopyOutResponse",
+	 /* I */ "EmptyQueryResponse",
+	 /* J */ 0,
+	 /* K */ "BackendKeyData",
+	 /* L */ 0,
+	 /* M */ 0,
+	 /* N */ "NoticeResponse",
+	 /* O */ 0,
+	 /* P */ 0,
+	 /* Q */ 0,
+	 /* R */ "Authentication",
+	 /* S */ "ParameterStatus",
+	 /* T */ "RowDescription",
+	 /* U */ 0,
+	 /* V */ "FunctionCallResponse",
+	 /* W */ "CopyBothResponse",
+	 /* X */ 0,
+	 /* Y */ 0,
+	 /* Z */ "ReadyForQuery",
+	 /* \x5b - \x5f */ 0, 0, 0, 0, 0,
+	 /* \x60 - \x6d */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* n */ "NoData",
+	 /* o */ 0,
+	 /* p */ 0,
+	 /* q */ 0,
+	 /* r */ 0,
+	 /* s */ "PortalSuspended",
+	 /* t */ "ParameterDescription",
+	 /* u */ 0,
+	 /* v */ "NegotiateProtocolVersion",
+};
+#define COMMAND_B_MIN ((unsigned char)1)
+#define COMMAND_B_MAX (sizeof(protocol_message_type_b) / sizeof(*protocol_message_type_b))
+
+static char *protocol_message_type_f[] = {
+	 /* B */ "Bind",
+	 /* C */ "Close",
+	 /* D */ "Describe",
+	 /* E */ "Execute",
+	 /* F */ "FunctionCall",
+	 /* G */ 0,
+	 /* H */ "Flush",
+	 /* I - O       */ 0, 0, 0, 0, 0, 0, 0,
+	 /* P */ "Parse",
+	 /* Q */ "Query",
+	 /* R */ 0,
+	 /* S */ "Sync",
+	 /* T - W       */ 0, 0, 0, 0,
+	 /* X */ "Terminate",
+	 /* Y */ 0,
+	 /* Z */ 0,
+	 /* \x5b - \x65 */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* f */ "CopyFail",
+	 /* g - o       */ 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	 /* p */ "AuthnenticationResponse",
+};
+#define COMMAND_F_MIN ((unsigned char)'B')
+#define COMMAND_F_MAX (sizeof(protocol_message_type_f) / sizeof(*protocol_message_type_f)) + COMMAND_F_MIN
+
+static char *protocol_message_type_bf[] = {
+	 /* c */ "CopyDone",
+	 /* d */ "CopyData",
+};
+#define COMMAND_BF_MIN ((unsigned char)'c')
+#define COMMAND_BF_MAX (sizeof(protocol_message_type_bf) / sizeof(*protocol_message_type_bf)) + COMMAND_BF_MIN
+
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void fputnbytes(FILE *f, const char *str, size_t n);
+static void pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void pqLogMsgString(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMsgnchar(PGconn *conn, const char *v, int length,
+						  PGCommSource commsource);
+static void pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void pqGetCurrentTime(char *currenttime);
+
+/*
+ * pqGetProtocolMsgType:
+ * 	Get a protocol type from first byte identifier
+ */
+static char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	char	   *message_type = 0;
+
+	if (c >= COMMAND_BF_MIN && c < COMMAND_BF_MAX)
+	{
+		message_type = protocol_message_type_bf[c - COMMAND_BF_MIN];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_BACKEND && c >= COMMAND_B_MIN && c < COMMAND_B_MAX)
+	{
+		message_type = protocol_message_type_b[c - COMMAND_B_MIN];
+		if (message_type)
+			return message_type;
+	}
+
+	if (commsource == FROM_FRONTEND && c >= COMMAND_F_MIN && c < COMMAND_F_MAX)
+	{
+		message_type = protocol_message_type_f[c - COMMAND_F_MIN];
+		if (message_type)
+			return message_type;
+	}
+
+	return "UnknownCommand";
+}
+
+/* pqInitMsgLog: Initializing logging message */
+static void
+pqInitMsgLog(PGconn *conn)
+{
+	conn->logging_message.state = LOG_FIRST_BYTE;
+	conn->logging_message.length = 0;
+}
+
+/* pqFreeFrontendEntry: forget stored protocol messages from frontend */
+static void
+pqFreeFrontendEntry(PGconn *conn)
+{
+	PGFrontendLogMsgEntry *entry;
+
+	entry = conn->frontend_entry_head;
+	while (entry != NULL)
+	{
+		PGFrontendLogMsgEntry *prev = entry;
+
+		entry = entry->next;
+		free(prev);
+	}
+	conn->frontend_entry_head = conn->frontend_entry_tail = NULL;
+}
+
+/* pqLogInvalidProtocol: Output a message the protocol is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqLogLineBreak:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+pqLogLineBreak(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqInitMsgLog(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg: Store message addresses that frontend sends.
+ *
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+	PGFrontendLogMsgEntry *newFrontendEntry;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		newFrontendEntry = (PGFrontendLogMsgEntry *) malloc(sizeof(PGFrontendLogMsgEntry));
+		if (newFrontendEntry)
+		{
+			newFrontendEntry->type = type;
+			newFrontendEntry->message_addr = conn->outMsgEnd - length;
+			newFrontendEntry->message_length = length;
+			newFrontendEntry->next = NULL;
+			if (conn->frontend_entry_tail)
+				conn->frontend_entry_tail->next = newFrontendEntry;
+			else
+				conn->frontend_entry_head = newFrontendEntry;
+			conn->frontend_entry_tail = newFrontendEntry;
+		}
+	}
+	else
+	{
+		/* Output one content per one line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fputnbytes(conn->Pfdebug, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg:
+ * 	Output frontend message contents after the message length.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn)
+{
+	int			message_addr;
+	int			length;
+	PGFrontendLogMsgEntry *entry;
+
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	entry = conn->frontend_entry_head;
+
+	for (entry = conn->frontend_entry_head;
+		 entry != NULL;
+		 entry = entry->next)
+	{
+		message_addr = entry->message_addr;
+		length = entry->message_length;
+
+		switch (entry->type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+							   length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+							  length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	pqFreeFrontendEntry(conn);
+	pqInitMsgLog(conn);
+}
+
+#define	TRACELOG_TIME_SIZE	33
+/*
+ * pqGetCurrentTime: get current time for trace log output
+ */
+static void
+pqGetCurrentTime(char *currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME	localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			 localTime.wYear, localTime.wMonth, localTime.wDay,
+			 localTime.wHour, localTime.wMinute, localTime.wSecond,
+			 localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm  *tm;
+	char		timezone[100];
+
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			 1900 + tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			 tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -98,12 +440,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -114,11 +455,55 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char	   *protocol_message_type;
+	char	   *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	char		current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_FIRST_BYTE:
+				pqGetCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s ", current_time, message_source);
+				/* If there is no first 1 byte protocol message, */
+				if (v == ' ')
+					return;
+
+				protocol_message_type = pqGetProtocolMsgType((unsigned char) v, commsource);
+				fprintf(conn->Pfdebug, "%s ", protocol_message_type);
+				/* Change the state to get protocol message length */
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqLogLineBreak(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -152,8 +537,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -181,12 +565,39 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
 
 /*
+ * pqLogMsgString: output a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
  */
@@ -202,11 +613,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, s, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -226,11 +633,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fputnbytes(conn->Pfdebug, conn->inBuffer + conn->inCursor, len);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -248,13 +651,39 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
+
+	return 0;
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fputnbytes(conn->Pfdebug, v, length);
+				fprintf(conn->Pfdebug, "\' ");
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
 	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fputnbytes(conn->Pfdebug, s, len);
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fputnbytes(conn->Pfdebug, v, length);
 		fprintf(conn->Pfdebug, "\n");
 	}
-
-	return 0;
 }
 
 /*
@@ -292,7 +721,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -314,11 +743,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -326,11 +759,66 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
+	return 0;
+}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	char	   *message_type = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+	int			message_addr;
 
-	return 0;
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+				/*
+				 * Output message type here for protocol messages that do not
+				 * have the first byte.
+				 */
+			case LOG_FIRST_BYTE:
+				if (conn->frontend_entry_head)
+				{
+					message_addr = conn->frontend_entry_head->message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->logging_message.state = LOG_LENGTH;
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				/* Change the state to log protocol message contents */
+				conn->logging_message.state = LOG_CONTENTS;
+				pqLogLineBreak(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
 }
 
 /*
@@ -548,8 +1036,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -585,7 +1072,7 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
 
@@ -594,6 +1081,12 @@ pqPutMsgEnd(PGconn *conn)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index c97841e..611d70a 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -123,6 +123,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a harf-finished logging message */
+			if (conn->Pfdebug)
+				pqLogLineBreak(msgLength, conn);
 			return;
 		}
 
@@ -156,7 +159,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a harf-finished logging message */
+				if (conn->Pfdebug)
+					pqLogLineBreak(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 64468ab..8a0ce35 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,53 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+}			PGCommSource;
+
+/* PGLogState defines the state of the Logging message state machine */
+typedef enum
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+}			PGLogState;
+
+/* Protocol message */
+typedef struct PGLogMsg
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+}			PGLogMsg;
+
+/* Frontend message data type */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+}			PGLogMsgDataType;
+
+/* Store frontend message address */
+typedef struct PGFrontendLogMsgEntry
+{
+	PGLogMsgDataType type;
+	int			message_addr;
+	int			message_length;
+	struct PGFrontendLogMsgEntry *next; /* list link */
+}			PGFrontendLogMsgEntry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -373,6 +420,13 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* Trace log info to output one line */
+	PGLogMsg	logging_message;
+
+	/* Trace log entrys needed when sending a message from frontend */
+	PGFrontendLogMsgEntry *frontend_entry_head; /* oldest stored frontend msg */
+	PGFrontendLogMsgEntry *frontend_entry_tail; /* newest stored frontend msg */
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -659,6 +713,7 @@ extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 						time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void pqLogLineBreak(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
#59Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: iwata.aya@fujitsu.com (#58)
2 attachment(s)
Re: libpq debug log

Hello Aya Iwata,

I like this patch, and I think we should have it. I have updated it, as
it had conflicts.

In 0002, I remove use of ftime(), because that function is obsolete.
However, with that change we lose printing of milliseconds in the trace
lines. I think that's not great, but I also think we can live without
them until somebody gets motivated to write the code for that. It seems
a little messy so I'd prefer to leave that for a subsequent commit.
(Maybe we can just use pg_strftime.)

Looking at the message type tables, I decided to get rid of the "bf"
table, which had only two bytes, and instead put CopyData and CopyDone
in the other two tables. That seems simpler. Also, the COMMAND_x_MAX
macros were a bit pointless; I just expanded at the only caller of each,
using lengthof(). And since the message type is unsigned, there's no
need to do "c >= 0" since it must always be true.

I added setlinebuf() to the debug file. Works better than without,
because "tail -f /tmp/libpqtrace.log" works as you'd expect.

One thing that it might be good to do is to avoid creating the message
type tables as const but instead initialize them if and only if tracing
is enabled, on the first call. I think that would not only save space
in the constant area of the library for the 99.99% of the cases where
tracing isn't used, but also make the initialization code be more
sensible (since presumably you wouldn't have to initialize all the
zeroes.)

Opinions? I experimented by patching psql as below (not intended for
commit) and it looks good. Only ErrorResponse prints the terminator as
a control character, or something:

2020-09-16 13:27:29.072 -03 < ErrorResponse 117 S "ERROR" V "ERROR" C "42704" M "no existe el slot de replicaci�n �foobar�" F "slot.c" L "408" R "ReplicationSlotAcquireInternal" ^@

diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 8232a0143b..01728ba8e8 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -301,6 +301,11 @@ main(int argc, char *argv[])

psql_setup_cancel_handler();

+	{
+		FILE *trace = fopen("/tmp/libpqtrace.log", "a+");
+		PQtrace(pset.db, trace);
+	}
+
 	PQsetNoticeProcessor(pset.db, NoticeProcessor, NULL);

SyncVariables();

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachments:

v8-0001-libpq-trace-v8.patchtext/x-diff; charset=us-asciiDownload
From ae0e3853a805d186abdd9e06acceca7ada48107a Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Wed, 16 Sep 2020 13:13:45 -0300
Subject: [PATCH v8 1/2] libpq trace v8

---
 doc/src/sgml/libpq.sgml             |   1 +
 src/interfaces/libpq/fe-connect.c   |   1 +
 src/interfaces/libpq/fe-misc.c      | 509 ++++++++++++++++++++++++++--
 src/interfaces/libpq/fe-protocol3.c |   8 +
 src/interfaces/libpq/libpq-int.h    |  52 +++
 src/tools/pgindent/typedefs.list    |   5 +
 6 files changed, 547 insertions(+), 29 deletions(-)

diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index a397073526..0d365f3bb3 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5854,6 +5854,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 724076a310..0795861c92 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6813,6 +6813,7 @@ PQtrace(PGconn *conn, FILE *debug_port)
 		return;
 	PQuntrace(conn);
 	conn->Pfdebug = debug_port;
+	setlinebuf(conn->Pfdebug);
 }
 
 void
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ff840b7730..de84da6e83 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,6 +35,7 @@
 
 #ifdef WIN32
 #include "win32.h"
+#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -45,6 +46,7 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
+#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -53,11 +55,304 @@
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
 
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x04 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void pqLogMsgString(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMsgnchar(PGconn *conn, const char *v, int length,
+						  PGCommSource commsource);
+static void pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void pqGetCurrentTime(char *currenttime);
+#define	TRACELOG_TIME_SIZE	33
+
+/*
+ * pqGetProtocolMsgType:
+ * 	Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	const char *message_type;
+
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		message_type = protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		message_type = protocol_message_type_f[c];
+	else
+		return "UnknownCommand";
+}
+
+/* pqInitMsgLog: Initializing logging message */
+static void
+pqInitMsgLog(PGconn *conn)
+{
+	conn->logging_message.state = LOG_FIRST_BYTE;
+	conn->logging_message.length = 0;
+}
+
+/* pqLogInvalidProtocol: Output a message the protocol is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->logging_message.state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqLogLineBreak:
+ * 	Check whether message formats is complete. If so,
+ * 	break the line.
+ */
+void
+pqLogLineBreak(int size, PGconn *conn)
+{
+	conn->logging_message.length -= size;
+	if (conn->logging_message.length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqInitMsgLog(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg: Store message addresses that frontend sends.
+ *
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->frontend_entry[conn->nMsgEntries].type = type;
+		conn->frontend_entry[conn->nMsgEntries].message_addr = conn->outMsgEnd - length;
+		conn->frontend_entry[conn->nMsgEntries].message_length = length;
+		conn->nMsgEntries++;
+	}
+	else
+	{
+		/* Output one content per one line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fwrite(conn->outBuffer + conn->outMsgEnd - length, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg:
+ * 	Output frontend message contents after the message length.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn)
+{
+	int			i;
+	int			message_addr;
+	int			length;
+
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	for (i = 0; i < conn->nMsgEntries; i++)
+	{
+		message_addr = conn->frontend_entry[i].message_addr;
+		length = conn->frontend_entry[i].message_length;
+
+		switch (conn->frontend_entry[i].type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+							   length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+							  length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	pqInitMsgLog(conn);
+}
+
+/*
+ * pqGetCurrentTime: get current time for trace log output
+ */
+static void
+pqGetCurrentTime(char *currenttime)
+{
+#ifdef WIN32
+	SYSTEMTIME	localTime;
+	TIME_ZONE_INFORMATION TimezoneInfo;
+
+	GetLocalTime(&localTime);
+
+	GetTimeZoneInformation(&TimezoneInfo);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			 localTime.wYear, localTime.wMonth, localTime.wDay,
+			 localTime.wHour, localTime.wMinute, localTime.wSecond,
+			 localTime.wMilliseconds, TimezoneInfo.Bias);
+#else
+	struct timeb localTime;
+	struct tm  *tm;
+	char		timezone[100];
+
+	ftime(&localTime);
+	tm = localtime(&localTime.time);
+
+	strftime(timezone, sizeof(timezone), "%Z", tm);
+	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
+			 1900 + tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
+			 tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
+#endif
+}
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,12 +380,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -101,11 +395,56 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	const char *protocol_message_type;
+	char	   *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	char		current_time[TRACELOG_TIME_SIZE];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_FIRST_BYTE:
+				pqGetCurrentTime(current_time);
+				fprintf(conn->Pfdebug, "%s %s ", current_time, message_source);
+				/* If there is no first 1 byte protocol message, */
+				if (v == ' ')
+					return;
+
+				protocol_message_type = pqGetProtocolMsgType((unsigned char) v, commsource);
+				fprintf(conn->Pfdebug, "%s ", protocol_message_type);
+				/* Change the state to get protocol message length */
+				conn->logging_message.state = LOG_LENGTH;
+				conn->logging_message.command = v;
+				conn->nMsgEntries = 0;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqLogLineBreak(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
 
 /*
  * pqGets[_append]:
@@ -139,8 +478,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -168,11 +506,38 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgString: output a a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
 /*
  * pqGetnchar:
  *	get a string of exactly len bytes in buffer s, no null termination
@@ -189,11 +554,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -213,11 +574,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -235,15 +592,42 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
 
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fwrite(v, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\' ");
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fwrite(v, 1, length, conn->Pfdebug);
+		fprintf(conn->Pfdebug, "\n");
+	}
+}
+
+
 /*
  * pqGetInt
  *	read a 2 or 4 byte integer and convert from network byte order
@@ -279,7 +663,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -301,11 +685,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -313,13 +701,69 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
-
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	char	   *message_type = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+	int			message_addr;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+				/*
+				 * Output message type here for protocol messages that do not
+				 * have the first byte.
+				 */
+			case LOG_FIRST_BYTE:
+				if (conn->nMsgEntries > 0)
+				{
+					message_addr = conn->frontend_entry[0].message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->logging_message.state = LOG_LENGTH;
+				/* fall-through */
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->logging_message.length = v - length;
+				/* Change the state to log protocol message contents */
+				conn->logging_message.state = LOG_CONTENTS;
+				pqLogLineBreak(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqLogLineBreak(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
+}
+
 /*
  * Make sure conn's output buffer can hold bytes_needed bytes (caller must
  * include already-stored data into the value!)
@@ -535,8 +979,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type ? msg_type : ' ', FROM_FRONTEND);
 
 	return 0;
 }
@@ -572,15 +1015,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 1696525475..2b8b9921ee 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -123,6 +123,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a half-finished logging message */
+			if (conn->Pfdebug)
+				pqLogLineBreak(msgLength, conn);
 			return;
 		}
 
@@ -156,7 +159,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqLogLineBreak(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1de91ae295..8af4163aa1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,52 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* PGLogState defines the state of the Logging message state machine */
+typedef enum
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+/* Protocol message */
+typedef struct PGLogMsg
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} PGLogMsg;
+
+/* Frontend message data type */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+/* Store frontend message address */
+typedef struct PGFrontendLogMsgEntry
+{
+	PGLogMsgDataType type;
+	int			message_addr;
+	int			message_length;
+} PGFrontendLogMsgEntry;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -376,6 +422,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* pending log message */
+	PGLogMsg	logging_message;
+	PGFrontendLogMsgEntry frontend_entry[MAXPGPATH];
+	int			nMsgEntries;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -674,6 +725,7 @@ extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
 						time_t finish_time);
 extern int	pqReadReady(PGconn *conn);
 extern int	pqWriteReady(PGconn *conn);
+extern void pqLogLineBreak(int size, PGconn *conn);
 
 /* === in fe-secure.c === */
 
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b1afb345c3..61f22404e8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1504,6 +1504,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1516,9 +1517,13 @@ PGEventResultCreate
 PGEventResultDestroy
 PGFInfoFunction
 PGFileType
+PGFrontendLogMsgEntry
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsg
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
-- 
2.20.1

v8-0002-Remove-use-of-obsolete-ftime.patchtext/x-diff; charset=us-asciiDownload
From ebed26d8e90744b46d2903a8f0449f748cdbeb1a Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Wed, 16 Sep 2020 17:06:32 -0300
Subject: [PATCH v8 2/2] Remove use of obsolete ftime()

Loses output of milliseconds :-(
---
 src/interfaces/libpq/fe-misc.c | 47 ++++++----------------------------
 1 file changed, 8 insertions(+), 39 deletions(-)

diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index de84da6e83..ee479d7863 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -35,7 +35,6 @@
 
 #ifdef WIN32
 #include "win32.h"
-#include <windows.h>
 #else
 #include <unistd.h>
 #include <sys/time.h>
@@ -46,7 +45,6 @@
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
-#include <sys/timeb.h>
 #endif
 
 #include "libpq-fe.h"
@@ -159,8 +157,6 @@ static void pqLogMsgString(PGconn *conn, const char *v, int length,
 static void pqLogMsgnchar(PGconn *conn, const char *v, int length,
 						  PGCommSource commsource);
 static void pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
-static void pqGetCurrentTime(char *currenttime);
-#define	TRACELOG_TIME_SIZE	33
 
 /*
  * pqGetProtocolMsgType:
@@ -322,38 +318,6 @@ pqLogFrontendMsg(PGconn *conn)
 	pqInitMsgLog(conn);
 }
 
-/*
- * pqGetCurrentTime: get current time for trace log output
- */
-static void
-pqGetCurrentTime(char *currenttime)
-{
-#ifdef WIN32
-	SYSTEMTIME	localTime;
-	TIME_ZONE_INFORMATION TimezoneInfo;
-
-	GetLocalTime(&localTime);
-
-	GetTimeZoneInformation(&TimezoneInfo);
-	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
-			 localTime.wYear, localTime.wMonth, localTime.wDay,
-			 localTime.wHour, localTime.wMinute, localTime.wSecond,
-			 localTime.wMilliseconds, TimezoneInfo.Bias);
-#else
-	struct timeb localTime;
-	struct tm  *tm;
-	char		timezone[100];
-
-	ftime(&localTime);
-	tm = localtime(&localTime.time);
-
-	strftime(timezone, sizeof(timezone), "%Z", tm);
-	snprintf(currenttime, TRACELOG_TIME_SIZE, "%4d-%02d-%02d %02d:%02d:%02d.%03d %s ",
-			 1900 + tm->tm_year, 1 + tm->tm_mon, tm->tm_mday,
-			 tm->tm_hour, tm->tm_min, tm->tm_sec, localTime.millitm, timezone);
-#endif
-}
-
 /*
  * PQlibVersion: return the libpq version number
  */
@@ -408,15 +372,20 @@ pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
 {
 	const char *protocol_message_type;
 	char	   *message_source = commsource == FROM_BACKEND ? "<" : ">";
-	char		current_time[TRACELOG_TIME_SIZE];
+	time_t		currtime;
+	struct tm  *tmp;
+	char		timestr[128];
 
 	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 	{
 		switch (conn->logging_message.state)
 		{
 			case LOG_FIRST_BYTE:
-				pqGetCurrentTime(current_time);
-				fprintf(conn->Pfdebug, "%s %s ", current_time, message_source);
+				currtime = time(NULL);
+				tmp = localtime(&currtime);
+				strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S %Z", tmp);
+
+				fprintf(conn->Pfdebug, "%s %s ", timestr, message_source);
 				/* If there is no first 1 byte protocol message, */
 				if (v == ' ')
 					return;
-- 
2.20.1

#60iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Alvaro Herrera (#59)
RE: libpq debug log

Hi Alvaro san,

Thank you for your update :)

Opinions? I experimented by patching psql as below (not intended for
commit) and it looks good. Only ErrorResponse prints the terminator as a
control character, or something:

I check code, changes and email. I agree with all of this.
I will review code, feature and performance if it is needed more closely.
(I'll start it next week.)

Regards,
Aya Iwata

Show quoted text

-----Original Message-----
From: Alvaro Herrera <alvherre@2ndquadrant.com>
Sent: Thursday, September 17, 2020 5:42 AM
To: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>
Cc: pgsql-hackers@postgresql.org; tgl@sss.pgh.pa.us;
robertmhaas@gmail.com; pchampion@pivotal.io; jdoty@pivotal.io;
raam.soft@gmail.com; Nagaura, Ryohei/永浦 良平
<nagaura.ryohei@fujitsu.com>; nagata@sraoss.co.jp;
peter.eisentraut@2ndquadrant.com; 'Kyotaro HORIGUCHI'
<horiguchi.kyotaro@lab.ntt.co.jp>; Jamison, Kirk/ジャミソン カーク
<k.jamison@fujitsu.com>
Subject: Re: libpq debug log

Hello Aya Iwata,

I like this patch, and I think we should have it. I have updated it, as it had
conflicts.

In 0002, I remove use of ftime(), because that function is obsolete.
However, with that change we lose printing of milliseconds in the trace lines.
I think that's not great, but I also think we can live without them until
somebody gets motivated to write the code for that. It seems a little messy
so I'd prefer to leave that for a subsequent commit.
(Maybe we can just use pg_strftime.)

Looking at the message type tables, I decided to get rid of the "bf"
table, which had only two bytes, and instead put CopyData and CopyDone in
the other two tables. That seems simpler. Also, the COMMAND_x_MAX
macros were a bit pointless; I just expanded at the only caller of each, using
lengthof(). And since the message type is unsigned, there's no need to do "c

= 0" since it must always be true.

I added setlinebuf() to the debug file. Works better than without, because
"tail -f /tmp/libpqtrace.log" works as you'd expect.

One thing that it might be good to do is to avoid creating the message type
tables as const but instead initialize them if and only if tracing is enabled, on
the first call. I think that would not only save space in the constant area of
the library for the 99.99% of the cases where tracing isn't used, but also make
the initialization code be more sensible (since presumably you wouldn't have
to initialize all the
zeroes.)

Opinions? I experimented by patching psql as below (not intended for
commit) and it looks good. Only ErrorResponse prints the terminator as a
control character, or something:

2020-09-16 13:27:29.072 -03 < ErrorResponse 117 S "ERROR" V "ERROR" C
"42704" M "no existe el slot de replicación «foobar»" F "slot.c" L "408" R
"ReplicationSlotAcquireInternal" ^@

diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c index
8232a0143b..01728ba8e8 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -301,6 +301,11 @@ main(int argc, char *argv[])

psql_setup_cancel_handler();

+	{
+		FILE *trace = fopen("/tmp/libpqtrace.log", "a+");
+		PQtrace(pset.db, trace);
+	}
+
PQsetNoticeProcessor(pset.db, NoticeProcessor, NULL);

SyncVariables();

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#61Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Alvaro Herrera (#59)
Re: libpq debug log

At Wed, 16 Sep 2020 17:41:55 -0300, Alvaro Herrera <alvherre@2ndquadrant.com> wrote in

Hello Aya Iwata,

I like this patch, and I think we should have it. I have updated it, as
it had conflicts.

In 0002, I remove use of ftime(), because that function is obsolete.
However, with that change we lose printing of milliseconds in the trace
lines. I think that's not great, but I also think we can live without
them until somebody gets motivated to write the code for that. It seems
a little messy so I'd prefer to leave that for a subsequent commit.
(Maybe we can just use pg_strftime.)

Looking at the message type tables, I decided to get rid of the "bf"
table, which had only two bytes, and instead put CopyData and CopyDone
in the other two tables. That seems simpler. Also, the COMMAND_x_MAX
macros were a bit pointless; I just expanded at the only caller of each,
using lengthof(). And since the message type is unsigned, there's no
need to do "c >= 0" since it must always be true.

I added setlinebuf() to the debug file. Works better than without,
because "tail -f /tmp/libpqtrace.log" works as you'd expect.

One thing that it might be good to do is to avoid creating the message
type tables as const but instead initialize them if and only if tracing
is enabled, on the first call. I think that would not only save space
in the constant area of the library for the 99.99% of the cases where
tracing isn't used, but also make the initialization code be more
sensible (since presumably you wouldn't have to initialize all the
zeroes.)

Opinions? I experimented by patching psql as below (not intended for
commit) and it looks good. Only ErrorResponse prints the terminator as
a control character, or something:

2020-09-16 13:27:29.072 -03 < ErrorResponse 117 S "ERROR" V "ERROR" C "42704" M "no existe el slot de replicación «foobar»" F "slot.c" L "408" R "ReplicationSlotAcquireInternal" ^@

diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 8232a0143b..01728ba8e8 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -301,6 +301,11 @@ main(int argc, char *argv[])

psql_setup_cancel_handler();

+	{
+		FILE *trace = fopen("/tmp/libpqtrace.log", "a+");
+		PQtrace(pset.db, trace);
+	}
+
PQsetNoticeProcessor(pset.db, NoticeProcessor, NULL);

SyncVariables();

I saw that version and have some comments.

+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	const char *message_type;

Compiler complains that this is unused.

+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
...
+	else
+		return "UnknownCommand";
+}

Compiler complains as "control reached end of non-void function"

+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->logging_message.state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqLogLineBreak(length, conn);

pqLogMsgString(conn, str, -1, FROM_*) means actual length may be
different from the caller thinks, but the pqLogLineBreak() subtracts
that value from the message length rememberd in in logging_message.
Anyway AFAICS the patch doesn't use the code path so we should remove
the first two lines.

+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);

That prints #65535 for v = -1 and length = 2. I think it should be
properly expanded as a signed integer.

@@ -139,8 +447,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
conn->inCursor = ++inCursor;

 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);

By the way, appendBinaryPQExpBuffer() enlarges its buffer by the size
of the exact length of the given data, but appends '\0' at the end of
the copied data. Couldn't that leads to an memory overrun?

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#62Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Kyotaro Horiguchi (#61)
Re: libpq debug log

On 2020-Oct-09, Kyotaro Horiguchi wrote:

I saw that version and have some comments.

+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	const char *message_type;

Compiler complains that this is unused.

+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
...
+	else
+		return "UnknownCommand";
+}

Compiler complains as "control reached end of non-void function"

Yeah, those two warnings are caused by the same problem, namely that I
was editing this function to make it simpler and apparently the patch
version I sent does not include all such changes. The fix is to remove
the message_type variable and have the two assignments be "return".

+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+

pqLogMsgString(conn, str, -1, FROM_*) means actual length may be
different from the caller thinks, but the pqLogLineBreak() subtracts
that value from the message length rememberd in in logging_message.
Anyway AFAICS the patch doesn't use the code path so we should remove
the first two lines.

True, +1 for removing it.

By the way, appendBinaryPQExpBuffer() enlarges its buffer by the size
of the exact length of the given data, but appends '\0' at the end of
the copied data. Couldn't that leads to an memory overrun?

Doesn't enlargePQExpBuffer() include room for the trailing zero? I
think it does.

#63Robert Haas
robertmhaas@gmail.com
In reply to: Alvaro Herrera (#59)
Re: libpq debug log

On Wed, Sep 16, 2020 at 4:41 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

2020-09-16 13:27:29.072 -03 < ErrorResponse 117 S "ERROR" V "ERROR" C "42704" M "no existe el slot de replicación «foobar»" F "slot.c" L "408" R "ReplicationSlotAcquireInternal" ^@

Ooh, that looks really nice. The ^@ at the end seems odd, though.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#64Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Alvaro Herrera (#62)
Re: libpq debug log

At Fri, 9 Oct 2020 11:48:59 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in

+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+

pqLogMsgString(conn, str, -1, FROM_*) means actual length may be
different from the caller thinks, but the pqLogLineBreak() subtracts
that value from the message length rememberd in in logging_message.
Anyway AFAICS the patch doesn't use the code path so we should remove
the first two lines.

True, +1 for removing it.

By the way, appendBinaryPQExpBuffer() enlarges its buffer by the size
of the exact length of the given data, but appends '\0' at the end of
the copied data. Couldn't that leads to an memory overrun?

Doesn't enlargePQExpBuffer() include room for the trailing zero? I
think it does.

Right. I faintly recall I said the same thing before..

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#65Michael Paquier
michael@paquier.xyz
In reply to: Kyotaro Horiguchi (#64)
Re: libpq debug log

On Wed, Oct 14, 2020 at 10:18:38AM +0900, Kyotaro Horiguchi wrote:

At Fri, 9 Oct 2020 11:48:59 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in

Doesn't enlargePQExpBuffer() include room for the trailing zero? I
think it does.

Right. I faintly recall I said the same thing before..

FWIW, as pqexpbuffer.c says, it does:
needed += str->len + 1; /* total space required now */
--
Michael

#66Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#60)
1 attachment(s)
Re: libpq debug log

Hello Aya Iwata

I've been hacking at this patch again. There were a few things I wasn't
too clear about, so I reordered the code and renamed the routines to try
to make it easier to follow.

One thing I didn't like very much is that all the structures and enums
were exposed to the world in libq-int.h. Because some enum members have
pretty generic names, I didn't like that much, so I moved the whole
thing to fe-misc.c, and renamed the structs. Also, the arrays don't
take space unless PQtrace() is called. (This is not what I was talking
about in my previous message. The idea I was trying to explain in my
previous message cannot possibly work, so I abandoned it.)

I also renamed functions to make it clear which handles frontend and
which handles backend. With that, it was pretty obvious that we had an
"reset BE message" in the routine to handle FE, and some clearing of FE
in the code that handles BE. I fixed things in a way that I think makes
the most sense.

I noticed that it's theoretically possible to have a FE message so large
(more than MAXPGPATH pieces) that it would overrun the array; so I added
a "print message" call after adding one piece, to avoid this. Also, why
MAXPGPATH? I added a new symbol MAX_FRONTEND_MSGS for this purpose.

There are some things still to do:

0. I added a XXX comment to pqFlush. Because we're storing messages in
fe_msgs that point to the libpq buffer, is it possible to end up with
trace messages that are pointing into outBuffer bytes that are already
sent, and perhaps even overwritten with newer bytes? Maybe not, but
it's unclear. Should we do pqLogFrontendMsg() preventively to avoid
this problem?

1. Is the handling of protocol 2 correct? Since it's completely
separate from protocol 3, I have not even looked at what it produces.
But the fact that pqLogMsgByte1 completely ignores the "commsource"
argument makes me suspect that it's not correct.
1a. How do we even test protocol 2 handling?

2. We need a mode to suppress print of time; this would be useful to
be able to test libpq at a low level. Maybe instead of time we can
print a monotonically-increasing packet sequence number. With this, we
could easily add tests for libpq itself. What user interface do we want
for this? Maybe we can add an "bits32 flags" parameter to PQtrace(),
with one bit for this use.

3. COPY ... (FORMAT BINARY) emits "invalid protocol" ... not good.

4. Even in text format, COPY output is not very useful. How can we
improve that?

5. Error messages are still printing the terminating zero byte. I
suggest that it should be suppressed.

6. Let's redefine pqTraceMaybeBreakLine() (I renamed from
pqLogLineBreak): If message is complete, print a newline; if message is
not complete, print a " ". Then, remove the whitespace after printing
each element. This should lead to lines that don't have an useless " "
at the end.

7. Why does it make sense to call pqTraceMaybeBreakLine when
commsource=FROM_FRONTEND?

Attachments:

v9-0001-libpq-trace.patchtext/x-diff; charset=us-asciiDownload
From 61e135006c3f48d6d5fb3e5a5400fca448e4cbc4 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Wed, 16 Sep 2020 13:13:45 -0300
Subject: [PATCH v9] libpq trace

---
 doc/src/sgml/libpq.sgml             |   1 +
 src/interfaces/libpq/fe-connect.c   |  12 +-
 src/interfaces/libpq/fe-misc.c      | 576 ++++++++++++++++++++++++++--
 src/interfaces/libpq/fe-protocol3.c |   8 +
 src/interfaces/libpq/libpq-int.h    |  15 +
 src/tools/pgindent/typedefs.list    |   5 +
 6 files changed, 588 insertions(+), 29 deletions(-)

diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index de60281fcb..bd6d89cf0f 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5844,6 +5844,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index b0ca37c2ed..856e011d9a 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6809,7 +6809,17 @@ PQtrace(PGconn *conn, FILE *debug_port)
 	if (conn == NULL)
 		return;
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (pqTraceInit(conn))
+	{
+		conn->Pfdebug = debug_port;
+		setlinebuf(conn->Pfdebug);
+	}
+	else
+	{
+		/* XXX report ENOMEM? */
+		fprintf(conn->Pfdebug, "Failed to initialize trace support\n");
+		fflush(conn->Pfdebug);
+	}
 }
 
 void
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 4ffc7f33fb..a6bb64dd5a 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -53,11 +53,154 @@
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+#define MAX_FRONTEND_MSGS 1024
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessage
+{
+	PGLogMsgDataType type;
+	int			message_addr;
+	int			message_length;
+} pqFrontendMessage;
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x04 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqLogFrontendMsg(PGconn *conn);
+static void pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void pqLogMsgString(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMsgnchar(PGconn *conn, const char *v, int length,
+						  PGCommSource commsource);
+
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,7 +228,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
@@ -101,7 +244,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +282,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -168,7 +310,7 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
@@ -189,11 +331,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -213,11 +351,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -235,11 +369,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +409,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -301,11 +431,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -313,10 +447,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
-
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -535,8 +665,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type, FROM_FRONTEND);
 
 	return 0;
 }
@@ -572,15 +701,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +737,383 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called mutiple
+ * times in a process; make sure it's idempotent.  We don't release memory
+ * on PQuntrace(), as that would be useless.
+ */
+bool
+pqTraceInit(PGconn *conn)
+{
+	/* already done? */
+	if (conn->be_msg != NULL)
+	{
+		conn->n_fe_msgs = 0;
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+		return true;
+	}
+
+	conn->be_msg = malloc(sizeof(pqBackendMessage));
+	if (conn->be_msg == NULL)
+		return false;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+
+	conn->fe_msg = malloc(MAX_FRONTEND_MSGS * sizeof(pqFrontendMessage));
+	if (conn->fe_msg == NULL)
+	{
+		free(conn->be_msg);
+		return false;
+	}
+	conn->n_fe_msgs = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		return protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		return protocol_message_type_f[c];
+	else
+		return "UnknownCommand";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Store message sent by frontend for later display.
+ *
+ *		In protocol v2, we immediately print each message as we receive it.
+ *		(XXX why?)
+ *		In protocol v3, we store the messages and print them all as a single
+ *		line when we get the message-end.
+ *
+ * XXX -- ??
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->fe_msg[conn->n_fe_msgs].type = type;
+		conn->fe_msg[conn->n_fe_msgs].message_addr = conn->outMsgEnd - length;
+		conn->fe_msg[conn->n_fe_msgs].message_length = length;
+		conn->n_fe_msgs++;
+		/* make sure not to overrun the buffer when a message is too large */
+		if (conn->n_fe_msgs >= MAX_FRONTEND_MSGS)
+			pqLogFrontendMsg(conn);
+	}
+	else
+	{
+		/* Output one content per line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fwrite(conn->outBuffer + conn->outMsgEnd - length, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn)
+{
+	int			i;
+	int			message_addr;
+	int			length;
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	for (i = 0; i < conn->n_fe_msgs; i++)
+	{
+		message_addr = conn->fe_msg[i].message_addr;
+		length = conn->fe_msg[i].message_length;
+
+		switch (conn->fe_msg[i].type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+							   length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+							  length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	conn->n_fe_msgs = 0;
+}
+
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char	   *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	time_t		currtime;
+	struct tm  *tmp;
+	char		timestr[128];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_FIRST_BYTE:
+				currtime = time(NULL);
+				tmp = localtime(&currtime);
+				strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S %Z", tmp);
+
+				fprintf(conn->Pfdebug, "%s %s ", timestr, message_source);
+				/* If there is no first 1 byte protocol message, */
+				if (v == '\0')
+					return;
+
+				fprintf(conn->Pfdebug, "%s ",
+						pqGetProtocolMsgType((unsigned char) v, commsource));
+				/* Next, log the message length */
+				conn->be_msg->state = LOG_LENGTH;
+				conn->be_msg->command = v;
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%c ", v);
+				pqTraceMaybeBreakLine(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	char	   *message_type = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+	int			message_addr;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_FIRST_BYTE:
+
+				/*
+				 * Output message type here for protocol messages that do not
+				 * have the first byte.
+				 */
+				if (conn->n_fe_msgs > 0)
+				{
+					message_addr = conn->fe_msg[0].message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->be_msg->state = LOG_LENGTH;
+				break;
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d ", v);
+				conn->be_msg->length = v - length;
+				/* Next, log the message contents */
+				conn->be_msg->state = LOG_CONTENTS;
+				pqTraceMaybeBreakLine(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d ", prefix, v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
+}
+
+
+/*
+ * pqLogMsgString: output a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\" ", v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fwrite(v, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\' ");
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fwrite(v, 1, length, conn->Pfdebug);
+		fprintf(conn->Pfdebug, "\n");
+	}
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,6 +1525,12 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
+	/*
+	 * XXX I don't think fflush here is sufficient: there could be unsent
+	 * trace messages pointing to the output area; may be overwritten after
+	 * this.  So we need to send stuff to the trace file before flushing the
+	 * libpq buffer.
+	 */
 	if (conn->Pfdebug)
 		fflush(conn->Pfdebug);
 
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 1696525475..1d4fa7840c 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -123,6 +123,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a half-finished logging message */
+			if (conn->Pfdebug)
+				pqTraceMaybeBreakLine(msgLength, conn);
 			return;
 		}
 
@@ -156,7 +159,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceMaybeBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1de91ae295..bc38fcfce8 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -376,6 +384,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* unwritten protocol traces */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
+	int			n_fe_msgs;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -668,6 +681,8 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn);
+extern void pqTraceMaybeBreakLine(int size, PGconn *conn);	/* XXX dubious */
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ff853634bc..7d39acb770 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1505,6 +1505,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1520,6 +1521,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3240,6 +3243,8 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
 pqbool
 pqsigfunc
 printQueryOpt
-- 
2.20.1

#67iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Alvaro Herrera (#66)
RE: libpq debug log

Hi Alvaro san,

Thank you for your email.
I will review this updated patch and I will let you know when I complete.
Please wait a moment.

Best regards,
Aya Iwata

Show quoted text

-----Original Message-----
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Sent: Tuesday, October 27, 2020 1:23 AM
To: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>
Cc: pgsql-hackers@postgresql.org; tgl@sss.pgh.pa.us;
robertmhaas@gmail.com; pchampion@pivotal.io; jdoty@pivotal.io;
raam.soft@gmail.com; Nagaura, Ryohei/永浦 良平
<nagaura.ryohei@fujitsu.com>; nagata@sraoss.co.jp;
peter.eisentraut@2ndquadrant.com; 'Kyotaro HORIGUCHI'
<horiguchi.kyotaro@lab.ntt.co.jp>; Jamison, Kirk/ジャミソン カーク
<k.jamison@fujitsu.com>
Subject: Re: libpq debug log

Hello Aya Iwata

I've been hacking at this patch again. There were a few things I wasn't too
clear about, so I reordered the code and renamed the routines to try to make it
easier to follow.

One thing I didn't like very much is that all the structures and enums were
exposed to the world in libq-int.h. Because some enum members have
pretty generic names, I didn't like that much, so I moved the whole thing to
fe-misc.c, and renamed the structs. Also, the arrays don't take space unless
PQtrace() is called. (This is not what I was talking about in my previous
message. The idea I was trying to explain in my previous message cannot
possibly work, so I abandoned it.)

I also renamed functions to make it clear which handles frontend and which
handles backend. With that, it was pretty obvious that we had an "reset BE
message" in the routine to handle FE, and some clearing of FE in the code that
handles BE. I fixed things in a way that I think makes the most sense.

I noticed that it's theoretically possible to have a FE message so large (more
than MAXPGPATH pieces) that it would overrun the array; so I added a "print
message" call after adding one piece, to avoid this. Also, why MAXPGPATH?
I added a new symbol MAX_FRONTEND_MSGS for this purpose.

There are some things still to do:

0. I added a XXX comment to pqFlush. Because we're storing messages in
fe_msgs that point to the libpq buffer, is it possible to end up with trace
messages that are pointing into outBuffer bytes that are already sent, and
perhaps even overwritten with newer bytes? Maybe not, but it's unclear.
Should we do pqLogFrontendMsg() preventively to avoid this problem?

1. Is the handling of protocol 2 correct? Since it's completely separate from
protocol 3, I have not even looked at what it produces.
But the fact that pqLogMsgByte1 completely ignores the "commsource"
argument makes me suspect that it's not correct.
1a. How do we even test protocol 2 handling?

2. We need a mode to suppress print of time; this would be useful to be able to
test libpq at a low level. Maybe instead of time we can print a
monotonically-increasing packet sequence number. With this, we could
easily add tests for libpq itself. What user interface do we want for this?
Maybe we can add an "bits32 flags" parameter to PQtrace(), with one bit for
this use.

3. COPY ... (FORMAT BINARY) emits "invalid protocol" ... not good.

4. Even in text format, COPY output is not very useful. How can we improve
that?

5. Error messages are still printing the terminating zero byte. I suggest that
it should be suppressed.

6. Let's redefine pqTraceMaybeBreakLine() (I renamed from
pqLogLineBreak): If message is complete, print a newline; if message is not
complete, print a " ". Then, remove the whitespace after printing each
element. This should lead to lines that don't have an useless " "
at the end.

7. Why does it make sense to call pqTraceMaybeBreakLine when
commsource=FROM_FRONTEND?

#68Greg Nancarrow
gregn4422@gmail.com
In reply to: Alvaro Herrera (#66)
Re: libpq debug log

On Tue, Oct 27, 2020 at 3:23 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

I've been hacking at this patch again. There were a few things I wasn't
too clear about, so I reordered the code and renamed the routines to try
to make it easier to follow.

Hi,

Hopefully Iwata-san will return to looking at this soon.

I have tried the latest patch a little (using "psql" as my client),
and have a few small comments so far:

- In pqTraceInit(), in the (admittedly rare) case that fe_msg malloc()
fails, I'd NULL out be_msg too after free(), rather than leave it
dangling (because if pgTraceInit() was ever invoked again, as the
comment says it could, it would result in previously freed memory
being accessed ...)

conn->fe_msg = malloc(MAX_FRONTEND_MSGS * sizeof(pqFrontendMessage));
if (conn->fe_msg == NULL)
{
free(conn->be_msg);
conn->be_msg = NULL;
return false;
}

- >3. COPY ... (FORMAT BINARY) emits "invalid protocol" ... not good.

That seemed to happen for me only if COPYing binary format to stdout.

UnknownCommand :::Invalid Protocol

- >5. Error messages are still printing the terminating zero byte. I

suggest that it should be suppressed.

Perhaps there's a more elegant way of doing it, but I got rid of the
logging of the zero byte using the following change to
pgLogMsgByte1(), though there still seems to be a trailing space
issue:

                        case LOG_CONTENTS:
-                               fprintf(conn->Pfdebug, "%c ", v);
+                               if (v != '\0')
+                                   fprintf(conn->Pfdebug, "%c ", v);
                                pqTraceMaybeBreakLine(sizeof(v), conn);
                                break;

Regards,
Greg Nancarrow
Fujitsu Australia

#69iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#60)
1 attachment(s)
RE: libpq debug log

Hi Alvaro san,

There are some things still to do:

I worked on some to do.

1. Is the handling of protocol 2 correct?

Protocol 3.0 is implemented in PostgreSQL v7.4 or later. Therefore, most servers and clients today want to connect using 3.0.
Also, wishful thinking, I think Protocol 2.0 will no longer be supported. So I didn't develop it aggressively.
Another reason I'm concerned about implementing it would make the code less maintainable.

5. Error messages are still printing the terminating zero byte. I
suggest that it should be suppressed.

I suppressed to print terminating zero byte like this;
2020-12-15 00:54:09 UTC < ErrorResponse 100 S "ERROR" V "ERROR" C "42P01" M "relation "a" does not exist" P "15" F "parse_relation.c" L "1379" R "parserOpenTable" 0

I thought about not outputting it, but the document said that if zero,
it was the last message, so I made it output "0".

6. Let's redefine pqTraceMaybeBreakLine() (I renamed from
pqLogLineBreak):

Sure. I redefined this function.

7. Why does it make sense to call pqTraceMaybeBreakLine when
commsource=FROM_FRONTEND?

Backend messages include break line. However frontend messages don’t have it. So I call this functions.

Pending lists.

0. XXX comments

There were 4 XXX comments in Alvaro san's patch. 3 XXX comments are left in current patch.
I will answer this in next e-mail.

2. We need a mode to suppress print of time;
3. COPY ... (FORMAT BINARY) emits "invalid protocol" ... not good.
4. Even in text format, COPY output is not very useful. How can we

improve that?

Regards,
Aya Iwata
Fujitsu

Attachments:

v10-0001-libpq-trace.patchapplication/octet-stream; name=v10-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 67c5d4c..0068854 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5859,6 +5859,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 7d04d36..30cab94 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6809,7 +6809,17 @@ PQtrace(PGconn *conn, FILE *debug_port)
 	if (conn == NULL)
 		return;
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (pqTraceInit(conn))
+	{
+		conn->Pfdebug = debug_port;
+		setlinebuf(conn->Pfdebug);
+	}
+	else
+	{
+		/* XXX report ENOMEM? */
+		fprintf(conn->Pfdebug, "Failed to initialize trace support\n");
+		fflush(conn->Pfdebug);
+	}
 }
 
 void
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 4ffc7f3..ff4c989 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -53,11 +53,154 @@
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+#define MAX_FRONTEND_MSGS 1024
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessage
+{
+	PGLogMsgDataType type;
+	int			message_addr;
+	int			message_length;
+} pqFrontendMessage;
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x04 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqLogFrontendMsg(PGconn *conn);
+static void pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource);
+static void pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void pqLogMsgString(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMsgnchar(PGconn *conn, const char *v, int length,
+						  PGCommSource commsource);
+
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,7 +228,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
@@ -101,7 +244,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +282,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -168,7 +310,7 @@ pqPuts(const char *s, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);
 
 	return 0;
 }
@@ -189,11 +331,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -213,11 +351,7 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 
 	conn->inCursor += len;
 
@@ -235,11 +369,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +409,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -301,11 +431,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT16, 2);
 			break;
 		case 4:
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
+			if (conn->Pfdebug)
+				pqStoreFrontendMsg(conn, LOG_INT32, 4);
 			break;
 		default:
 			pqInternalNotice(&conn->noticeHooks,
@@ -313,10 +447,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 							 (unsigned long) bytes);
 			return EOF;
 	}
-
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -535,8 +665,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqLogMsgByte1(conn, msg_type, FROM_FRONTEND);
 
 	return 0;
 }
@@ -572,15 +701,23 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
+	if (conn->Pfdebug && PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+	{
 		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
 				conn->outMsgEnd - conn->outCount);
+	}
 
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+		{
+			pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+			pqLogFrontendMsg(conn);
+		}
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +737,389 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called mutiple
+ * times in a process; make sure it's idempotent.  We don't release memory
+ * on PQuntrace(), as that would be useless.
+ */
+bool
+pqTraceInit(PGconn *conn)
+{
+	/* already done? */
+	if (conn->be_msg != NULL)
+	{
+		conn->n_fe_msgs = 0;
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+		return true;
+	}
+
+	conn->be_msg = malloc(sizeof(pqBackendMessage));
+	if (conn->be_msg == NULL)
+		return false;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+
+	conn->fe_msg = malloc(MAX_FRONTEND_MSGS * sizeof(pqFrontendMessage));
+	if (conn->fe_msg == NULL)
+	{
+		free(conn->be_msg);
+		return false;
+	}
+	conn->n_fe_msgs = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		return protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		return protocol_message_type_f[c];
+	else
+		return "UnknownCommand";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Store message sent by frontend for later display.
+ *
+ *		In protocol v2, we immediately print each message as we receive it.
+ *		(XXX why?)
+ *		In protocol v3, we store the messages and print them all as a single
+ *		line when we get the message-end.
+ *
+ * XXX -- ??
+ * 	Message length is added at the last if message is sent by the frontend.
+ * 	To arrange the log output format, frontend message contents are stored in the list.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->fe_msg[conn->n_fe_msgs].type = type;
+		conn->fe_msg[conn->n_fe_msgs].message_addr = conn->outMsgEnd - length;
+		conn->fe_msg[conn->n_fe_msgs].message_length = length;
+		conn->n_fe_msgs++;
+		/* make sure not to overrun the buffer when a message is too large */
+		if (conn->n_fe_msgs >= MAX_FRONTEND_MSGS)
+			pqLogFrontendMsg(conn);
+	}
+	else
+	{
+		/* Output one content per line in older protocol version */
+		switch (type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> %c\n", message);
+				break;
+
+			case LOG_STRING:
+				memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+				fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);
+				break;
+
+			case LOG_NCHAR:
+				fprintf(conn->Pfdebug, "To backend (%d)> ", length);
+				fwrite(conn->outBuffer + conn->outMsgEnd - length, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\n");
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh16(result16);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + conn->outMsgEnd - length, length);
+				result = (int) pg_ntoh32(result32);
+				fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn)
+{
+	int			i;
+	int			message_addr;
+	int			length;
+	char		message;
+	uint16		result16 = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+
+	for (i = 0; i < conn->n_fe_msgs; i++)
+	{
+		message_addr = conn->fe_msg[i].message_addr;
+		length = conn->fe_msg[i].message_length;
+
+		switch (conn->fe_msg[i].type)
+		{
+			case LOG_BYTE1:
+				memcpy(&message, conn->outBuffer + message_addr, length);
+				pqLogMsgByte1(conn, message, FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+							   length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+							  length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh16(result16);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT32:
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result = (int) pg_ntoh32(result32);
+				pqLogMsgInt(conn, result, length, FROM_FRONTEND);
+				break;
+		}
+	}
+	conn->n_fe_msgs = 0;
+}
+
+/*
+ * pqLogMsgByte1: output 1 char message to the log
+ */
+static void
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char	   *message_source = commsource == FROM_BACKEND ? "<" : ">";
+	time_t		currtime;
+	struct tm  *tmp;
+	char		timestr[128];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_FIRST_BYTE:
+				currtime = time(NULL);
+				tmp = localtime(&currtime);
+				strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S %Z", tmp);
+
+				fprintf(conn->Pfdebug, "%s %s ", timestr, message_source);
+				/* If there is no first 1 byte protocol message, return */
+				if (v == '\0')
+					return;
+
+				fprintf(conn->Pfdebug, "%s ",
+						pqGetProtocolMsgType((unsigned char) v, commsource));
+				/* Next, log the message length */
+				conn->be_msg->state = LOG_LENGTH;
+				conn->be_msg->command = v;
+				break;
+
+			case LOG_CONTENTS:
+				/* Suppresses printing terminating zero byte */
+				if (v == '\0')
+					fprintf(conn->Pfdebug, "0");
+				else
+					fprintf(conn->Pfdebug, "%c", v);
+				pqTraceMaybeBreakLine(sizeof(v), conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "FROM backend> %c\n", v);
+
+	return;
+}
+
+/*
+ * pqLogMsgInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	char	   *message_type = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+	int			message_addr;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_FIRST_BYTE:
+
+				/*
+				 * Output message type here for protocol messages that do not
+				 * have the first byte.
+				 */
+				if (conn->n_fe_msgs > 0)
+				{
+					message_addr = conn->fe_msg[0].message_addr;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownCommand";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+				conn->be_msg->state = LOG_LENGTH;
+				break;
+
+			case LOG_LENGTH:
+				fprintf(conn->Pfdebug, "%d", v);
+				conn->be_msg->length = v - length;
+				/* Next, log the message contents */
+				conn->be_msg->state = LOG_CONTENTS;
+				pqTraceMaybeBreakLine(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d", prefix, v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);
+}
+
+
+/*
+ * pqLogMsgString: output a null-terminated string to the log
+ */
+static void
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (length < 0)
+		length = strlen(v) + 1;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\"", v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", v);
+}
+
+/*
+ * pqLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				fwrite(v, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\'");
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, "From backend (%d)> ", length);
+		fwrite(v, 1, length, conn->Pfdebug);
+		fprintf(conn->Pfdebug, "\n");
+	}
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,8 +1531,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
+	/*
+	 * There could be unsent trace messages pointing to the output area; may be
+	 * overwritten after this.  So we need to send stuff to the trace file
+	 * before flushing the libpq buffer.
+	 */
 	if (conn->Pfdebug)
+	{
+		pqLogFrontendMsg(conn);
 		fflush(conn->Pfdebug);
+	}
 
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 1696525..1d4fa78 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -123,6 +123,9 @@ pqParseInput3(PGconn *conn)
 				 */
 				handleSyncLoss(conn, id, msgLength);
 			}
+			/* Terminate a half-finished logging message */
+			if (conn->Pfdebug)
+				pqTraceMaybeBreakLine(msgLength, conn);
 			return;
 		}
 
@@ -156,7 +159,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceMaybeBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1de91ae..bc38fcf 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -376,6 +384,11 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* unwritten protocol traces */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
+	int			n_fe_msgs;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -668,6 +681,8 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn);
+extern void pqTraceMaybeBreakLine(int size, PGconn *conn);	/* XXX dubious */
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a9dca71..54de754 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1506,6 +1506,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1521,6 +1522,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3253,6 +3256,8 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
 pqbool
 pqsigfunc
 printQueryOpt
#70k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#69)
RE: libpq debug log

On Tuesday, December 15, 2020 8:12 PM, Iwata-san wrote:

There are some things still to do:

I worked on some to do.

Hi Iwata-san,

Thank you for updating the patch.
I would recommend to register this patch in the upcoming commitfest
to help us keep track of it. I will follow the thread to provide more
reviews too.

1. Is the handling of protocol 2 correct?

Protocol 3.0 is implemented in PostgreSQL v7.4 or later. Therefore, most
servers and clients today want to connect using 3.0.
Also, wishful thinking, I think Protocol 2.0 will no longer be supported. So I
didn't develop it aggressively.
Another reason I'm concerned about implementing it would make the code
less maintainable.

Though I have also not tested it with 2.0 server yet, do I have to download 7.3
version to test that isn't it? Silly question, do we still want to have this
feature for 2.0?
I understand that protocol 2.0 is still supported, but it is only used for
PostgreSQL versions 7.3 and earlier, which is not updated by fixes anymore
since we only backpatch up to previous 5 versions. However I am not sure if
it's a good idea, but how about if we just support this feature for protocol 3.
There are similar chunks of code in fe-exec.c, PQsendPrepare(), PQsendDescribe(),
etc. that already do something similar, so I just thought in hindsight if we can
do the same.
if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
{
<new code here>
}
else
{
/* Protocol 2.0 isn't supported */
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext("function requires at least protocol version 3.0\n"));
return 0;
}

But if it's necessary to provide this improved trace output for 2.0 servers, then ignore what
I suggested above, and although difficult I think we should test for protocol 2.0 in older server.

Some minor code comments I noticed:
1.
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								 * protocol message type */

"identifing" should be "identifying". And closing */ should be on 3rd line.

2.
+			case LOG_CONTENTS:
+				/* Suppresses printing terminating zero byte */

--> Suppress printing of terminating zero byte

Regards,
Kirk Jamison

#71tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: k.jamison@fujitsu.com (#70)
RE: libpq debug log

From: k.jamison@fujitsu.com <k.jamison@fujitsu.com>

I understand that protocol 2.0 is still supported, but it is only used for
PostgreSQL versions 7.3 and earlier, which is not updated by fixes anymore
since we only backpatch up to previous 5 versions. However I am not sure if
it's a good idea, but how about if we just support this feature for protocol 3.

+1
I thought psqlODBC (still) allows the user to choose the protocol version, it looks like the current psqlODBC disallows pre-3 protocol:

[libpq_connect()]
MYLOG(0, "libpq connection to the database established.\n");
pversion = PQprotocolVersion(pqconn);
if (pversion < 3)
{
MYLOG(0, "Protocol version %d is not supported\n", pversion);
goto cleanup;
}

There are similar chunks of code in fe-exec.c, PQsendPrepare(),
PQsendDescribe(),
etc. that already do something similar, so I just thought in hindsight if we can
do the same.
if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
{
<new code here>
}
else
{
/* Protocol 2.0 isn't supported */
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext("function requires at least protocol
version 3.0\n"));
return 0;
}

I haven't looked at the patch and don't know the code structure, but I want the code clutter to be minimized. So, I think we should avoid putting if statements like above here and there.

Plus, I don't think it's not necessary to fail the processing when the protocol version is 2; we can just stop the trace output. So, how about disabling the trace output silently if the protocol turns out to be < 3 upon connection?

Regards
Takayuki Tsunakawa

#72Masahiko Sawada
sawada.mshk@gmail.com
In reply to: k.jamison@fujitsu.com (#70)
Re: libpq debug log

Iwata-san,

On Mon, Dec 21, 2020 at 5:20 PM k.jamison@fujitsu.com
<k.jamison@fujitsu.com> wrote:

On Tuesday, December 15, 2020 8:12 PM, Iwata-san wrote:

There are some things still to do:

I worked on some to do.

Hi Iwata-san,

Thank you for updating the patch.
I would recommend to register this patch in the upcoming commitfest
to help us keep track of it. I will follow the thread to provide more
reviews too.

1. Is the handling of protocol 2 correct?

Protocol 3.0 is implemented in PostgreSQL v7.4 or later. Therefore, most
servers and clients today want to connect using 3.0.
Also, wishful thinking, I think Protocol 2.0 will no longer be supported. So I
didn't develop it aggressively.
Another reason I'm concerned about implementing it would make the code
less maintainable.

Though I have also not tested it with 2.0 server yet, do I have to download 7.3
version to test that isn't it? Silly question, do we still want to have this
feature for 2.0?
I understand that protocol 2.0 is still supported, but it is only used for
PostgreSQL versions 7.3 and earlier, which is not updated by fixes anymore
since we only backpatch up to previous 5 versions. However I am not sure if
it's a good idea, but how about if we just support this feature for protocol 3.
There are similar chunks of code in fe-exec.c, PQsendPrepare(), PQsendDescribe(),
etc. that already do something similar, so I just thought in hindsight if we can
do the same.
if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
{
<new code here>
}
else
{
/* Protocol 2.0 isn't supported */
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext("function requires at least protocol version 3.0\n"));
return 0;
}

But if it's necessary to provide this improved trace output for 2.0 servers, then ignore what
I suggested above, and although difficult I think we should test for protocol 2.0 in older server.

Some minor code comments I noticed:
1.
+       LOG_FIRST_BYTE,                         /* logging the first byte identifing the
+                                                                * protocol message type */

"identifing" should be "identifying". And closing */ should be on 3rd line.

2.
+                       case LOG_CONTENTS:
+                               /* Suppresses printing terminating zero byte */

--> Suppress printing of terminating zero byte

The patch got some review comments a couple weeks ago but the patch
was still marked as "needs review", which was incorrect. Also cfbot[1]http://commitfest.cputube.org/
is unhappy with this patch. So I'm switching the patch as "waiting on
author".

Regards,

[1]: http://commitfest.cputube.org/

--
Masahiko Sawada
EnterpriseDB: https://www.enterprisedb.com/

#73matsumura.ryo@fujitsu.com
matsumura.ryo@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#69)
1 attachment(s)
RE: libpq debug log

Hi Iwata-san

I reviewed v10-0001-libpq-trace.patch. (But I don't check recent discussion...)
I found some bugs.
I'm suggesting some refactoring.

****
@@ -6809,7 +6809,17 @@ PQtrace(PGconn *conn, FILE *debug_port)
+       if (pqTraceInit(conn))
+       {
+               conn->Pfdebug = debug_port;
+               setlinebuf(conn->Pfdebug);

If debug_port is NULL, setlinebuf() causes segmentation fault.
We should have policy that libpq-trace framework works only when
Pfdebug is not NULL.

For example, we should also think that PQtrace(conn, NULL) has same
effect as PQuntrace(conn).

****
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x65 ... \0x6d */

There should be 9 zero, but 15 zero.

****
@@ -85,7 +228,7 @@ pqGetc(char *result, PGconn *conn)
    if (conn->Pfdebug)
-       fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+       pqLogMsgByte1(conn, *result, FROM_BACKEND);

I suggest to move confirming Pfdebug to pqLogMsgByte1() and
other logging functions. If you want to avoid overhead of calling
function, use macro function like the following:

#define pqLogMsgByte1(CONN, CH, SOURCE) \
((CONN)->Pfdebug ? pqLogMsgByte1(CONN, CH, SOURCE) : 0)

****
@@ -168,7 +310,7 @@ pqPuts(const char *s, PGconn *conn)
    if (pqPutMsgBytes(s, strlen(s) + 1, conn))
        return EOF;
    if (conn->Pfdebug)
-       fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+       pqStoreFrontendMsg(conn, LOG_STRING, strlen(s) + 1);

It's better that you store strlen(s) to local variable and use it.
strlen(s) is not cost-free.

****
@@ -301,11 +431,15 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
            tmp2 = pg_hton16((uint16) value);
            if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
                return EOF;
+           if (conn->Pfdebug)
+               pqStoreFrontendMsg(conn, LOG_INT16, 2);
            break;
        case 4:
            tmp4 = pg_hton32((uint32) value);
            if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
                return EOF;
+           if (conn->Pfdebug)
+               pqStoreFrontendMsg(conn, LOG_INT32, 4);
            break

It's better to make the style same as pqGetInt().
(It is not important.)

switch(bytes) {
case 2:
type = LOG_INT16;
break;
case 4:
type = LOG_INT32;
break;
}
pqStoreFrontendMsg(conn, type, bytes);

****
+pqTraceInit(PGconn *conn)
+{
+   conn->fe_msg = malloc(MAX_FRONTEND_MSGS * sizeof(pqFrontendMessage));
+   if (conn->fe_msg == NULL)
+   {
+       free(conn->be_msg);
+       return false;

Maybe, we need to clear conn->be_msg by NULL for next calling pqTraceInit().

****
+pqTraceInit(PGconn *conn)
+{
+   conn->fe_msg = malloc(MAX_FRONTEND_MSGS * sizeof(pqFrontendMessage));
+   conn->n_fe_msgs = 0;

Data structure design is odd.
Frontend message is basically constructed as the following:
- message type
- message length
- field
- field
- field

So the deisign may be as the follwong and remove n_fe_msg from pg_conn.

typedef struct pqFrontendMessageField
{
int offset_in_buffer; message_addr is not real address.
int length; message_length is wrong name. it is length of FIELD.
} pqFrontendMessageField;

typedef struct pqFrontendMessage
{
PGLogMsgDataType type;
int field_num;
pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
} pqFrontendMessage;

And then pqTraceInit() is as the following.
conn->fe_msg = malloc(sizeof(pqFrontendMessage) +
MAX_FRONTEND_MSG_FIELDS * sizeof(pqFrontendMessage));
conn->fe_msg->field_num = 0;

****
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+       return "UnknownCommand";

It is "UnknownMessageType", isn't it?

****
+ * XXX -- ??
+ *     Message length is added at the last if message is sent by the frontend.
+ *     To arrange the log output format, frontend message contents are stored in the list.

I understand as the following:
* The message length is fixed after putting the last field,
* but message length should be print before printing any field.
* So we must store the field data in memory.

****
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+           case LOG_STRING:
+               memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);
+               fprintf(conn->Pfdebug, "To backend> \"%c\"\n", message);

%s is correct.
At least, %c is not correct.

****
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+           case LOG_BYTE1:
+               memcpy(&message, conn->outBuffer + conn->outMsgEnd - length, length);

It is unnecessary to call memcpy().
LOG_BYTE1, LOG_STRING, and LOG_NCHAR can be passed its pointer directly to fprintf().
You can also pass LOG_INT* data with casting to fprintf() without memcpy(), but I think either is fine.

****
+           case LOG_INT16:
+               fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);
+           case LOG_INT32:
+               fprintf(conn->Pfdebug, "To backend (#%d)> %c\n", length, result);

%d is correct.
At least, %c is not correct.

****
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+   char        message;
+   uint16      result16 = 0;
+   uint32      result32 = 0;
+   int         result = 0;

It's better that these variables declared in each block.
Initializing result* is unnecessary.

case LOG_INT16:
{
uint16 wk;
memcpy(&wk, conn->outBuffer + conn->outMsgEnd - 2, 2);
wk = pg_ntoh16(wk);
fprintf(conn->Pfdebug, "To backend (#%d)> %h\n", 2, wk);
break;
}

****
+pqLogFrontendMsg(PGconn *conn)
+   int         message_addr;
+   int         length;
+   char        message;
+   uint16      result16 = 0;
+   uint32      result32 = 0;
+   int         result = 0;

Same as pqStoreFrontendMsg().

+ memcpy(&message, conn->outBuffer + message_addr, length);

Same as pqStoreFrontendMsg().

****
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+   char       *message_source = commsource == FROM_BACKEND ? "<" : ">";

I understand that 'commsource' means source, but message_source means direction of sending.
It is better that the variable is named message_direction.

****
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+            case LOG_FIRST_BYTE:
+               if (v == '\0')
+                   return;

Maybe is it needed for packets whose msg_type is '\0'?

I get the following output because pqLogMsgnchar() is called
at conn->be_msg->state == LENGTH. I attach a program for reproduction.

2020-12-25 00:58:48 UTC > StartupMessage :::Invalid Protocol

I think confusing transition of state causes it.
I suggest refactoring instead of fixing the above bug.

msg_type of Startup packet, SSLRequest packet, and GSSNegotiate packet
is '\0'. (I guess GSSNegotiate packet may be forgotten).
At these packet, Byte1 is not sent actually, but if libpq-trace framework consider
that it is sent, the transition may become more clear like the following:

pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
:
fprintf(conn->Pfdebug, "%s %s ", timestr, message_source);
/*
* True type of message tagged '\0' is known when next 4 bytes is checked.
* So we delay printing message type to pqLogMsgInt().
*/
if (v != '\0')
fprintf(conn->Pfdebug, "%s ",
pqGetProtocolMsgType((unsigned char) v, commsource));

conn->be_msg->state = LOG_LENGTH;
conn->be_msg->command = v;

pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
:

/* remove case LOG_FIRST_BYTE and... */

case LOG_LENGTH:
if (conn->be_msg->command == '\0')
{
/* We delayed to print message type for special message. */
message_addr = conn->fe_msg[0].message_addr;
memcpy(&result32, conn->outBuffer + message_addr, 4);
result = (int) pg_ntoh32(result32);

if (result == NEGOTIATE_SSL_CODE)
message_type = "SSLRequest";
else if (result == NEGOTIATE_GSS_CODE)
message_type = "GSSRequest";
else
message_type = "StartupMessage";
fprintf(conn->Pfdebug, "%s ", message_type);
}

fprintf(conn->Pfdebug, "%d", v);
conn->be_msg->length = v - length;
conn->be_msg->state = LOG_CONTENTS;
pqTraceMaybeBreakLine(0, conn);
break;

****
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+           case LOG_CONTENTS:
+               /* Suppresses printing terminating zero byte */
+               if (v == '\0')
+                   fprintf(conn->Pfdebug, "0");

Is the above code for end of 'E' or 'N' message?
We should comment because it is special case.
Additionally, we may confuse whether 0 is numerical character or '\0'.

/*
* 'E' or 'N' message format is as the following:
* 'E' len [byte1-tag string(null-terminated)]* \0(eof-tag)
*
* So we detect eof-tag at pqLogMsgByte1() with LOG_CONTENTS state.
* The eof-tag is data so we should print it.
* On the way, we care other non-ascii character.
*/
if (!isascii(v))
fprintf(conn->Pfdebug, "\\x%02x", v);

****
+pqLogMsgByte1(PGconn *conn, char v, PGCommSource commsource)
+   if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+   else
+       fprintf(conn->Pfdebug, "FROM backend> %c\n", v);

Umm. Also see the following.

+pqLogMsgInt(PGconn *conn, int v, int length, PGCommSource commsource)
+   if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+   else
+       fprintf(conn->Pfdebug, "To backend (#%d)> %d\n", length, v);

In the later case, "FROM backend" is correct.

The bug is caused by confusing design.
I suggest to refactor that:
- Move callings fprintf() in pqStoreFrontendMsg() to each pqLogMsgXXX()
- pqStoreFrontendMsg() calls each pqLogMsgXXX().

****
+pqLogMsgString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+   if (length < 0)
+       length = strlen(v) + 1;

I cannot find length==-1 route.

****
+pqLogMsgnchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+               fprintf(conn->Pfdebug, "\'");
+               fwrite(v, 1, length, conn->Pfdebug);
+               fprintf(conn->Pfdebug, "\'");

We should consider that 'v' is binary data.
So it is better to convert non-ascii character to ascii (e.g. x%02x format).

e.g. case of StartupPacket
2020-12-25 08:07:42 UTC > StartupMessage 40 'userk5userdatabasetemplate1'

The length of StartupPacket includes length itselft.
The length of 'user...tepmlate1' is only 27.
We could not find 8 bytes(40 - 4 - 27 = 8).

If non-ascii character is printed, we can find them.
2020-12-25 08:29:19 UTC > StartupMessage 40 '\x00\x03\x00\x00user\x00k5user\x00database\x00template1\x00\x00'

static void
pqLogMsgBinary(PGconn *conn, const char *v, int length, PGCommSource commsource)
{
int i, pin;
for (pin = i = 0; i < length; ++i)
{
if (isprint(v[i]))
continue;
else
{
fwrite(v + pin, 1, i - pin, conn->Pfdebug);
fprintf(conn->Pfdebug, "\\x%02x", v[i]);
pin = i + 1;
}
}
if (pin < length)
fwrite(v + pin, 1, length - pin, conn->Pfdebug);
}

****
@@ -123,6 +123,9 @@ pqParseInput3(PGconn *conn)
+           /* Terminate a half-finished logging message */
+           if (conn->Pfdebug)
+               pqTraceMaybeBreakLine(msgLength, conn);
            return;

I understand that the above code only want to do [conn->be_msg->length = 0].
If it is true, you should create new wrapper function like pqTraceForcelyTerminateMessage()
and handleSyncLoss() itself should call pqTraceForcelyTerminateMessage().

pqTraceForcelyTerminateMessage()
{
if (conn->be_msg->length > 0)
fprintf(conn->Pfdebug, "\n");

pqTraceResetBeMsg(conn);
}

****
pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
if (conn->Pfdebug)
fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
name, value);

I think the above code has become to be unnecessary because new pqTrace framework
become to print it.

Regards
Ryo Matsumura

Attachments:

app.ctext/plain; name=app.cDownload
#74iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: matsumura.ryo@fujitsu.com (#73)
1 attachment(s)
RE: libpq debug log

Hi,

Thank you for your review. I modified the code in response to those reviews.

This patch includes these items:
- Fix the code according to reviews
- Fix COPY output issue
- Change to not support Protocol v2.0
It is rarely used anymore and de-support it makes code more simpler. Please see the discussion in this thread for more details.

Regards,
Aya Iwata

Attachments:

v11-0001-libpq-trace.patchapplication/octet-stream; name=v11-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 2bb3bf7..4b2bd31 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5882,6 +5882,7 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
     <listitem>
      <para>
       Enables  tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 2b78ed8..025be0e 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6764,8 +6764,23 @@ PQtrace(PGconn *conn, FILE *debug_port)
 {
 	if (conn == NULL)
 		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (pqTraceInit(conn))
+	{
+		conn->Pfdebug = debug_port;
+		if (conn->Pfdebug != NULL)
+			setlinebuf(conn->Pfdebug);
+	}
+	else
+	{
+		/* XXX report ENOMEM? */
+		fprintf(conn->Pfdebug, "Failed to initialize trace support\n");
+		fflush(conn->Pfdebug);
+	}
 }
 
 void
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..8bc9966 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -53,11 +53,174 @@
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	FROM_BACKEND,
+	FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								   protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+#define MAX_FRONTEND_MSGS 1024
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessageField
+{
+	int 		offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	PGLogMsgDataType type;
+	int			field_num;
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x04 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqLogFrontendMsg(PGconn *conn);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogMessageByte1(PGconn *conn, char v, PGCommSource commsource);
+static void pqLogMessageInt(PGconn *conn, int v, int length, PGCommSource commsource);
+static void pqLogMessageString(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+						  PGCommSource commsource);
+
+#define pqLogMsgByte1(CONN, CH, SOURCE) \
+((CONN)->Pfdebug ? pqLogMessageByte1(CONN, CH, SOURCE) : 0)
+
+#define pqLogMsgInt(CONN, INT, LENGTH, SOURCE) \
+((CONN)->Pfdebug ? pqLogMessageInt(CONN, INT, LENGTH, SOURCE) : 0)
+
+#define pqLogMsgString(CONN, CH, LENGTH, SOURCE) \
+((CONN)->Pfdebug ? pqLogMessageString(CONN, CH, LENGTH, SOURCE) : 0)
+
+#define pqLogMsgnchar(CONN, CH, LENGTH, SOURCE) \
+((CONN)->Pfdebug ? pqLogMessagenchar(CONN, CH, LENGTH, SOURCE) : 0)
 
 /*
  * PQlibVersion: return the libpq version number
@@ -84,8 +247,7 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	pqLogMsgByte1(conn, *result, FROM_BACKEND);
 
 	return 0;
 }
@@ -101,7 +263,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -138,9 +300,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+	pqLogMsgString(conn, buf->data, buf->len + 1, FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +324,12 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int length = strlen(s) + 1;
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -188,12 +349,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+	pqLogMsgnchar(conn, s, len, FROM_BACKEND);
 
 	return 0;
 }
@@ -212,15 +368,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+	pqLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len, FROM_BACKEND);
 	conn->inCursor += len;
-
 	return 0;
 }
 
@@ -235,11 +384,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -278,8 +423,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	pqLogMsgInt(conn, *result, (unsigned int) bytes, FROM_BACKEND);
 
 	return 0;
 }
@@ -294,15 +438,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType	type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +462,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -520,8 +667,6 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 		/* allow room for message length */
 		endPos += 4;
 	}
-	else
-		lenPos = -1;
 
 	/* make sure there is room for message header */
 	if (pqCheckOutBufferSpace(endPos, conn))
@@ -534,9 +679,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+	pqLogMsgByte1(conn, msg_type, FROM_FRONTEND);
 
 	return 0;
 }
@@ -572,15 +715,15 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		pqLogMsgInt(conn, (int) msgLen, 4, FROM_FRONTEND);
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +743,374 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called mutiple
+ * times in a process; make sure it's idempotent.  We don't release memory
+ * on PQuntrace(), as that would be useless.
+ */
+bool
+pqTraceInit(PGconn *conn)
+{
+	/* already done? */
+	if (conn->be_msg != NULL)
+	{
+		if (conn->fe_msg)
+			conn->fe_msg->field_num = 0;
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+		return true;
+	}
+
+	conn->be_msg = malloc(sizeof(pqBackendMessage));
+	if (conn->be_msg == NULL)
+		return false;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+
+	conn->fe_msg = malloc(sizeof(pqFrontendMessage) +
+							MAX_FRONTEND_MSGS * sizeof(pqFrontendMessageField));
+	conn->fe_msg->field_num = 0;
+	if (conn->fe_msg == NULL)
+	{
+		free(conn->be_msg);
+		/* NULL out for the case that fe_msg malloc fails */
+		conn->be_msg = NULL;
+		return false;
+	}
+	conn->fe_msg->field_num = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		return protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		return protocol_message_type_f[c];
+	else
+		return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Store message sent by frontend for later display.
+ *
+ *		We store the messages and print them all as a single line
+ *		when we get the message-end.
+ *
+ *		The message length is fixed after putting the last field, but message
+ *		length should be print before printing any fields.So we must store the
+ *		field data in memory.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		conn->fe_msg->type = type;
+		conn->fe_msg->fields[conn->fe_msg->field_num].offset_in_buffer = conn->outMsgEnd - length;
+		conn->fe_msg->fields[conn->fe_msg->field_num].length = length;
+		conn->fe_msg->field_num++;
+		/* make sure not to overrun the buffer when a message is too large */
+		if (conn->fe_msg->field_num >= MAX_FRONTEND_MSGS)
+			pqLogFrontendMsg(conn);
+	}
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn)
+{
+	int			i;
+	int			message_addr;
+	int			length;
+
+	for (i = 0; i < conn->fe_msg->field_num; i++)
+	{
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		switch (conn->fe_msg[i].type)
+		{
+			case LOG_BYTE1:
+				pqLogMsgByte1(conn, (char)*(conn->outBuffer + message_addr), FROM_FRONTEND);
+				break;
+
+			case LOG_STRING:
+				pqLogMsgString(conn, conn->outBuffer + message_addr,
+							   length, FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMsgnchar(conn, conn->outBuffer + message_addr,
+							  length, FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+				uint16		result16;
+				memcpy(&result16, conn->outBuffer + message_addr, length);
+				result16 = pg_ntoh16(result16);
+				pqLogMsgInt(conn, result16, length, FROM_FRONTEND);
+				break;
+				}
+
+			case LOG_INT32:
+				{
+				uint32		result32;
+				memcpy(&result32, conn->outBuffer + message_addr, length);
+				result32 = pg_ntoh32(result32);
+				pqLogMsgInt(conn, result32, length, FROM_FRONTEND);
+				break;
+				}
+		}
+	}
+	conn->fe_msg->field_num = 0;
+}
+
+/*
+ * pqLogMessageByte1: output 1 char message to the log
+ */
+static void
+pqLogMessageByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char	   *message_direction = commsource == FROM_BACKEND ? "<" : ">";
+	time_t		currtime;
+	struct tm  *tmp;
+	char		timestr[128];
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_FIRST_BYTE:
+				currtime = time(NULL);
+				tmp = localtime(&currtime);
+				strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S %Z", tmp);
+
+				fprintf(conn->Pfdebug, "%s %s ", timestr, message_direction);
+				/*
+				 * True type of message tagged '\0' is known when next 4 bytes is
+				 * checked. So we delay printing message type to pqLogMsgInt()
+				 */
+				if (v != '\0')
+					fprintf(conn->Pfdebug, "%s ",
+							pqGetProtocolMsgType((unsigned char) v, commsource));
+				/* Next, log the message length */
+				conn->be_msg->state = LOG_LENGTH;
+				conn->be_msg->command = v;
+				break;
+
+			case LOG_CONTENTS:
+				/*
+				 *  'E' or 'N' message format is as the following:
+				 *  'E' len [byte1-tag string(null-terminated)]* \0(eof-tag)
+				 *  So we detect eof-tag at pqLogMsgByte1() with LOG_CONTENTS state.
+				 *  The eof-tag is data so we should print it.
+				 *  On the way, we care other non-ascii character.
+				 */
+				if (!isprint(v))
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				else
+					fprintf(conn->Pfdebug, "%c", v);
+				pqTraceMaybeBreakLine(1, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+	return;
+}
+
+/*
+ * pqLogMessageInt: output a 2 or 4 bytes integer message to the log
+ */
+static void
+pqLogMessageInt(PGconn *conn, int v, int length, PGCommSource commsource)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	char	   *message_type = 0;
+	uint32		result32 = 0;
+	int			result = 0;
+	int			message_addr;
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_LENGTH:
+				if (conn->be_msg->command == '\0')
+				{
+					/* We delayed to print message type for special message. */
+					if (conn->fe_msg->field_num > 0)
+					{
+						message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+						memcpy(&result32, conn->outBuffer + message_addr, 4);
+						result = (int) pg_ntoh32(result32);
+
+						if (result == NEGOTIATE_SSL_CODE)
+							message_type = "SSLRequest";
+						else if (result == NEGOTIATE_GSS_CODE)
+							message_type = "GSSRequest";
+						else
+							message_type = "StartupMessage";
+					}
+					else
+						message_type = "UnknownMessage";
+					fprintf(conn->Pfdebug, "%s ", message_type);
+				}
+				fprintf(conn->Pfdebug, "%d", v);
+				conn->be_msg->length = v - length;
+				/* Next, log the message contents */
+				conn->be_msg->state = LOG_CONTENTS;
+				pqTraceMaybeBreakLine(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d", prefix, v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+static void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\"%s\"", v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+}
+
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	int i, pin;
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+		if (pin < length)
+			fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+	}
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMessagenchar(PGconn *conn, const char *v, int length, PGCommSource commsource)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "\'");
+				if (!isprint(v[0]))
+					pqLogBinaryMsg(conn, v, length, commsource);
+				else
+					fwrite(v, 1, length, conn->Pfdebug);
+				fprintf(conn->Pfdebug, "\'");
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+			default:
+				pqLogInvalidProtocol(conn);
+				break;
+		}
+	}
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,8 +1522,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
+	/*
+	 * There could be unsent trace messages pointing to the output area; may be
+	 * overwritten after this.  So we need to send stuff to the trace file
+	 * before flushing the libpq buffer.
+	 */
 	if (conn->Pfdebug)
+	{
+		pqLogFrontendMsg(conn);
 		fflush(conn->Pfdebug);
+	}
 
 	if (conn->outCount > 0)
 		return pqSendSome(conn, conn->outCount);
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..d1a74f1 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +288,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +365,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +377,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +407,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +471,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1640,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db4983..e3de550 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -376,6 +384,10 @@ struct pg_conn
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
 
+	/* unwritten protocol traces */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
+
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
 
@@ -672,6 +684,8 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn);
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fb57b83..6854260 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1521,6 +1521,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1536,6 +1537,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3269,6 +3272,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#75tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#74)
RE: libpq debug log

From: iwata.aya@fujitsu.com <iwata.aya@fujitsu.com>

This patch includes these items:

Is there anything else in this revision that is not handled?

Below are my comments.

Also, why don't you try running the regression tests with a temporary modification to PQtrace() to output the trace to a file? The sole purpose is to confirm that this patch makes the test crash (core dump).

(1)
-	conn->Pfdebug = debug_port;
+	if (pqTraceInit(conn))
+	{
+		conn->Pfdebug = debug_port;
+		if (conn->Pfdebug != NULL)
+			setlinebuf(conn->Pfdebug);
+	}
+	else
+	{
+		/* XXX report ENOMEM? */
+		fprintf(conn->Pfdebug, "Failed to initialize trace support\n");
+		fflush(conn->Pfdebug);
+	}
 }

* When the passed debug_port is NULL, the function should return after calling PQuntrace().

* Is setlinebuf() available on Windows? Maybe you should use setvbuf() instead.

* I don't see the need for separate pqTraceInit() function, because it is only called here.

(2)
+bool
+pqTraceInit(PGconn *conn)
+{
+	/* already done? */
+	if (conn->be_msg != NULL)
+	{

What's this code for? I think it's sufficient that PQuntrace() frees b_msg and PQtrace() always allocates b_msg.

(3)
+	conn->fe_msg = malloc(sizeof(pqFrontendMessage) +
+							MAX_FRONTEND_MSGS * sizeof(pqFrontendMessageField));
+	conn->fe_msg->field_num = 0;
+	if (conn->fe_msg == NULL)
+	{
+		free(conn->be_msg);
+		/* NULL out for the case that fe_msg malloc fails */
+		conn->be_msg = NULL;
+		return false;
+	}
+	conn->fe_msg->field_num = 0;

The memory for the fields array is one entry larger than necessary. The allocation code would be:

+	conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							MAX_FRONTEND_MSGS * sizeof(pqFrontendMessageField));

Also, the line with "field_num = 0" appears twice. The first one should be removed.

(4)
+static void
+pqLogMessageByte1(PGconn *conn, char v, PGCommSource commsource)
+{
...
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{

I think you can remove the protocol version check in various logging functions, because PQtrace() disables logging when the protocol version is 2.

(5)
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
pgParameterStatus *pstatus;
pgParameterStatus *prev;

- if (conn->Pfdebug)
- fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
- name, value);
-

Where is this information output instead?

(6)
+ * Set up state so that we can trace. NB -- this might be called mutiple

mutiple -> multiple

(7)
+	LOG_FIRST_BYTE,				/* logging the first byte identifing the
+								   protocol message type */

identifing -> identifying

(8)
+#define pqLogMsgByte1(CONN, CH, SOURCE) \
+((CONN)->Pfdebug ? pqLogMessageByte1(CONN, CH, SOURCE) : 0)

For functions that return void, follow the style of CHECK_FOR_INTERRUPTS() defined in miscadmin.h.

(9)
+				currtime = time(NULL);
+				tmp = localtime(&currtime);
+				strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S %Z", tmp);
+
+				fprintf(conn->Pfdebug, "%s %s ", timestr, message_direction);

It's better to have microsecond accuracy, because knowing the accumulation of such fine-grained timings may help to troubleshoot heavily-loaded client cases. You can mimic setup_formatted_log_time() in src/backend/utils/error/elog.c. This is used for the %m marker in log_line_prefix.

(10)
I think it's better to use tabs (or any other character that is less likely to appear in the log field) as a field separator, because it makes it easier to process the log with a spreadsheet or some other tools.

(11)
+				/*
+				 * True type of message tagged '\0' is known when next 4 bytes is
+				 * checked. So we delay printing message type to pqLogMsgInt()
+				 */
+				if (v != '\0')
+					fprintf(conn->Pfdebug, "%s ",
+							pqGetProtocolMsgType((unsigned char) v, commsource));

In what cases does '\0' appear as a message type?

(12)
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		return protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		return protocol_message_type_f[c];
+	else
+		return "UnknownMessage";
+}

This function returns NULL (=0) when protocol_message_type_b/f[c] is 0. That crashes the caller:

+					fprintf(conn->Pfdebug, "%s ",
+							pqGetProtocolMsgType((unsigned char) v, commsource));

Plus, you may as well print the invalid message type. Why don't you do something like this:

+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	static char unknown_message[64];
+	char *msg = NULL;
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		msg = protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		msg = protocol_message_type_f[c];
+
+	if (msg == NULL)
+	{
+		sprintf(unknown_message, "Unknown message %02x", c);
+		msg = unknown_message;
+	}
+
+	return msg;
+}
(13)
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}

/*
* Unexpected message in IDLE state; need to recover somehow.

What's this situation like? Why output a new line and reset the trace status?

(14)
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn)
+{
+	fprintf(conn->Pfdebug, ":::Invalid Protocol\n");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}

Is it sufficient to just reset the state field? Isn't it necessary to call pqTraceResetBeMsg() instead?

(15)
@@ -212,15 +368,8 @@ pqSkipnchar(size_t len, PGconn *conn)
...
conn->inCursor += len;
-
return 0;
}

This is an unnecessary removal of an empty line.

(16)
+static void
+pqLogMessageByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char	   *message_direction = commsource == FROM_BACKEND ? "<" : ">";
...
+		switch (conn->be_msg->state)

This function handles messages in both directions. But the switch condition is based on the backend message state (b_msg). How is this correct?

(17)
What's the difference between pqLogMsgString() and pqLogMsgnchar()? They both take a length argument. Do they have to be separate functions?

(18)
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);

To match the style for backend messages with pqLogMsgByte1 etc., why don't you wrap the conn->Pfdebug check in the macro?

(19)
@@ -520,8 +667,6 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
/* allow room for message length */
endPos += 4;
}
- else
- lenPos = -1;

Why is this change necessary?

(20)
+static void
+pqLogFrontendMsg(PGconn *conn)
...
+	for (i = 0; i < conn->fe_msg->field_num; i++)
+	{
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		switch (conn->fe_msg[i].type)
+		{
+			case LOG_BYTE1:

When I saw this switch condition, the confusion exploded. Shouldn't the type should be the attribute of each field as follows?

+ switch (conn->fe_msg->fields[i].type)

I also found the function names for frontend -> backend messages hard to grasp. Could you show the flow of function calls when sending messages to the backend?

(21)
+				uint16		result16;
+				memcpy(&result16, conn->outBuffer + message_addr, length);

You should have an empty line between the declaration part and execution part.

(22)
I couldn't understand pqLogBinaryMsg() at all. Could you explain what it does? Particularly, if all bytes in the string is printable, the function seems to do nothing:

+ if (isprint(v[i]))
+ continue;

Should the following part be placed outside the for loop? What does 'pin' mean? (I want the variable name to be more intuitive.)

+		if (pin < length)
+			fwrite(v + pin, 1, length - pin, conn->Pfdebug);

Regards
Takayuki Tsunakawa

#76kuroda.hayato@fujitsu.com
kuroda.hayato@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#75)
RE: libpq debug log

Dear Tsunakawa-san, Iwata-san,

* Is setlinebuf() available on Windows? Maybe you should use setvbuf() instead.

Yeah, cfbot2021 throws errors:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.124922

```
src/interfaces/libpq/fe-connect.c(6776): warning C4013: 'setlinebuf' undefined; assuming extern returning int [C:\projects\postgresql\libpq.vcxproj]
```

The manpage of setlinebuf() suggests how to replace it, so you should follow.

```
The setbuffer() function is the same, except that the size of the buffer is up to the caller, rather than being determined by the
default BUFSIZ. The setlinebuf() function is exactly equivalent to the call:

setvbuf(stream, NULL, _IOLBF, 0);
```

Hayato Kuroda
FUJITSU LIMITED

#77k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#75)
RE: libpq debug log

Hi Iwata-san,

In addition to Tsunakawa-san's comments,
The compiler also complains:
fe-misc.c:678:20: error: ‘lenPos’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
conn->outMsgStart = lenPos;

There's no need for variable lenPos anymore since we have decided *not* to support pre protocol 3.0.
And by that we have to update the description of pqPutMsgStart too.
So you can remove the lenPos variable and the condition where you have to check for protocol version.

diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 8bc9966..3de48be 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -644,14 +644,12 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
  *
  * The state variable conn->outMsgStart points to the incomplete message's
  * length word: it is either outCount or outCount+1 depending on whether
- * there is a type byte.  If we are sending a message without length word
- * (pre protocol 3.0 only), then outMsgStart is -1.  The state variable
- * conn->outMsgEnd is the end of the data collected so far.
+ * there is a type byte.  The state variable conn->outMsgEnd is the end of
+ * the data collected so far.
  */
 int
 pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 {
-       int                     lenPos;
        int                     endPos;

/* allow room for message type byte */
@@ -661,9 +659,8 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
endPos = conn->outCount;

        /* do we want a length word? */
-       if (force_len || PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+       if (force_len)
        {
-               lenPos = endPos;
                /* allow room for message length */
                endPos += 4;
        }
@@ -675,7 +672,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
        if (msg_type)
                conn->outBuffer[conn->outCount] = msg_type;
        /* set up the message pointers */
-       conn->outMsgStart = lenPos;
+       conn->outMsgStart = endPos;
        conn->outMsgEnd = endPos;

At the same time, the one below lacks one more zero. (Only has 8)
There should be 9 as Matsumura-san mentioned.
+ 0, 0, 0, 0, 0, 0, 0, 0, /* \x65 ... \0x6d */

The following can be removed in pqStoreFrontendMsg():
+ *		In protocol v2, we immediately print each message as we receive it.
+ *		(XXX why?)
Maybe the following description can be paraphrased:
+ *		The message length is fixed after putting the last field, but message
+ *		length should be print before printing any fields.So we must store the
+ *		field data in memory.
to:
+ *		The message length is fixed after putting the last field. But message
+ *		length should be printed before printing any field, so we must store
+ *		the field data in memory.
In pqStoreFrontendMsg, pqLogMessagenchar, pqLogMessageString,
pqLogMessageInt, pqLogMessageByte1, maybe it is unneccessary to use
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
because you have already indicated in the PQtrace() to return
silently when protocol 2.0 is detected.
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;

In pqLogMessageInt(),
+ /* We delayed to print message type for special message. */
can be paraphrased to:
/* We delay printing of the following special message_type */

Regards,
Kirk Jamison

#78Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#75)
1 attachment(s)
Re: libpq debug log

On 2021-Jan-17, tsunakawa.takay@fujitsu.com wrote:

* I don't see the need for separate pqTraceInit() function, because it is only called here.

That's true, but it'd require that we move PQtrace() to fe-misc.c,
because pqTraceInit() uses definitions that are private to that file.

(2)
+bool
+pqTraceInit(PGconn *conn)
+{
+	/* already done? */
+	if (conn->be_msg != NULL)
+	{

What's this code for? I think it's sufficient that PQuntrace() frees b_msg and PQtrace() always allocates b_msg.

The point is to be able to cope with a connection that calls PQtrace()
multiple times, with or without an intervening PQuntrace().
We'd make no friends if we made PQtrace() crash, or leak memory if
called a second time ...

(5)
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
pgParameterStatus *pstatus;
pgParameterStatus *prev;

- if (conn->Pfdebug)
- fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
- name, value);
-

Where is this information output instead?

Hmm, seems to have been lost. I restored it, but didn't verify
the resulting behavior carefully.

(9)
+				currtime = time(NULL);
+				tmp = localtime(&currtime);
+				strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S %Z", tmp);
+
+				fprintf(conn->Pfdebug, "%s %s ", timestr, message_direction);

It's better to have microsecond accuracy, because knowing the accumulation of such fine-grained timings may help to troubleshoot heavily-loaded client cases. You can mimic setup_formatted_log_time() in src/backend/utils/error/elog.c. This is used for the %m marker in log_line_prefix.

Yeah, it was me that removed printing of milliseconds -- Iwata-san's
original patch used ftime() which is not portable. I had looked for
portable ways to get this but couldn't find anything that didn't involve
linking backend files, so I punted. But looking again now, it is quite
simple: just use standard strftime and localtime and just concatenate
both together. Similar to what setup_formatted_log_time except we don't
worry about the TZ at all.

(10)
I think it's better to use tabs (or any other character that is less likely to appear in the log field) as a field separator, because it makes it easier to process the log with a spreadsheet or some other tools.

I can buy that. I changed it to have some tabs; the lines are now:

timestamp "> or <" <tab> "message type" <tab> length <space> message contents

I think trying to apply tabs inside the message contents is going to be
more confusing than helpful.

(12)
[...]
Plus, you may as well print the invalid message type. Why don't you do something like this:

+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	static char unknown_message[64];
+	char *msg = NULL;
+	if (commsource == FROM_BACKEND && c < lengthof(protocol_message_type_b))
+		msg = protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c < lengthof(protocol_message_type_f))
+		msg = protocol_message_type_f[c];
+
+	if (msg == NULL)
+	{
+		sprintf(unknown_message, "Unknown message %02x", c);
+		msg = unknown_message;
+	}
+
+	return msg;
+}

Right. I had written something like this, but in the end decided not to
bother because it doesn't seem terribly important. You can't do exactly
what you suggest, because it has the problem that you're returning a
stack-allocated variable even though your stack is about to be blown
away. My copy had a static buffer that was malloc()ed on first use (and
if allocation fails, return a const string). Anyway, I fixed the
crasher problem.

The protocol_message_type_b array had the serious problem it confused
the entry for byte 1 (0x01) with that of char '1' (and 2 and 3), so it
printed a lot of 'Unknown message' lines. Clearly, the maintenance of
this array is going to be a pain point of this patch (counting number of
zeroes is no fun), and I think we're going to have some way to have an
explicit initializer, where we can do things like
protocol_message_type_b['A'] = "NotificationResponse";
etc instead of the current way, which is messy, hard to maintain.
I'm not sure how to do that and not make things worse, however.

(16)
+static void
+pqLogMessageByte1(PGconn *conn, char v, PGCommSource commsource)
+{
+	char	   *message_direction = commsource == FROM_BACKEND ? "<" : ">";
...
+		switch (conn->be_msg->state)

This function handles messages in both directions. But the switch condition is based on the backend message state (b_msg). How is this correct?

It's not.

I split things so that this function only prints backend messages; when
frontend messages are to be printed, we use a single fprintf() instead.
See about (20), below.

(17)
What's the difference between pqLogMsgString() and pqLogMsgnchar()? They both take a length argument. Do they have to be separate functions?

nchar are not null-terminated and we escape !isprint() characters. I'm
not sure that the difference is significant enough, but I'm also not
sure if they can be merged into one.

(18)
if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);

To match the style for backend messages with pqLogMsgByte1 etc., why don't you wrap the conn->Pfdebug check in the macro?

I think these macros are useless and should be removed. There's more
verbosity and confusion caused by them, than the clarity they provide.

(20)
+static void
+pqLogFrontendMsg(PGconn *conn)
...
+	for (i = 0; i < conn->fe_msg->field_num; i++)
+	{
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		switch (conn->fe_msg[i].type)
+		{
+			case LOG_BYTE1:

When I saw this switch condition, the confusion exploded. Shouldn't the type should be the attribute of each field as follows?

+ switch (conn->fe_msg->fields[i].type)

I also found the function names for frontend -> backend messages hard to grasp. Could you show the flow of function calls when sending messages to the backend?

Exactly! I was super confused about this too, and eventually decided to
rewrite the whole thing so that the 'type' is in the Fields struct. And
while doing that, I noticed that it would make more sense to let the
fe_msg array be realloc'ed if it gets full, rather than making it fixed
size. This made me add pqTraceUninit(), so that we can free the array
if it has grown, to avoid reserving arbitrarily large amounts of memory
after PQuntrace() on a session that traced large messages.

(22)
I couldn't understand pqLogBinaryMsg() at all. Could you explain what it does? Particularly, if all bytes in the string is printable, the function seems to do nothing:

+ if (isprint(v[i]))
+ continue;

Should the following part be placed outside the for loop? What does 'pin' mean? (I want the variable name to be more intuitive.)

+		if (pin < length)
+			fwrite(v + pin, 1, length - pin, conn->Pfdebug);

Yes :-( I fixed that too.

This patch needs more work still, of course.

--
�lvaro Herrera Valdivia, Chile
"Java is clearly an example of money oriented programming" (A. Stepanov)

Attachments:

v12-0001-libpq-trace.patchtext/x-diff; charset=us-asciiDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 2bb3bf77e4..9fad769b0c 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,19 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);
 </synopsis>
      </para>
 
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>(flags & TRACE_SUPPRESS_TIMESTAMPS)</literal> is
+      true, then timestamps are not printed with each message.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583aa9..5bc0a610c0 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6761,12 +6761,28 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 }
 
 void
-PQtrace(PGconn *conn, FILE *debug_port)
+PQtrace(PGconn *conn, FILE *debug_port, bits32 flags)
 {
 	if (conn == NULL)
 		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (!debug_port)
+		return;
+	if (pqTraceInit(conn, flags))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
 }
 
 void
@@ -6779,6 +6795,8 @@ PQuntrace(PGconn *conn)
 		fflush(conn->Pfdebug);
 		conn->Pfdebug = NULL;
 	}
+	pqTraceUninit(conn);
+	conn->traceFlags = 0;
 }
 
 PQnoticeReceiver
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6acd89..25efa47bc3 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -51,13 +51,168 @@
 #include "libpq-int.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
+#include "pgtime.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqStoreFeMsgStart(PGconn *conn, char type);
+static void pqLogFrontendMsg(PGconn *conn, int msgLen);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogMessageByte1(PGconn *conn, char v);
+static void pqLogMessageInt(PGconn *conn, int v, int length);
+static void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,7 +240,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMessageByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +256,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +294,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMessageString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -157,18 +311,19 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 	return pqGets_internal(buf, conn, false);
 }
 
-
 /*
  * pqPuts: write a null-terminated string to the current message
  */
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +344,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMessagenchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,14 +364,9 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqLogMessagenchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
-
 	return 0;
 }
 
@@ -235,11 +381,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +421,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMessageInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +436,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +460,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -473,8 +618,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 	}
 
 	/* realloc failed. Probably out of memory */
-	appendPQExpBufferStr(&conn->errorMessage,
-						 "cannot allocate memory for input buffer\n");
+	appendPQExpBuffer(&conn->errorMessage,
+					  "cannot allocate memory for input buffer\n");
 	return EOF;
 }
 
@@ -535,8 +680,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqStoreFeMsgStart(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +716,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +743,471 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+bool
+pqTraceInit(PGconn *conn, bits32 flags)
+{
+	conn->traceFlags = flags;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(sizeof(pqFrontendMessage) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+
+	return true;
+}
+
+/*
+ * Deallocate frontend-message tracking memory.  We only do this because
+ * it can grow arbitrarily large, and skip it in the initial state, because
+ * it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		pfree(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+		conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		/* realloc if we've exceeded available space */
+		if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+		{
+			if (conn->fe_msg->max_fields > INT_MAX / 2)
+			{
+				fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+				PQuntrace(conn);
+			}
+			conn->fe_msg =
+				realloc(conn->fe_msg,
+						sizeof(pqFrontendMessage) +
+						2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+			if (conn->fe_msg == NULL)
+			{
+				fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+				PQuntrace(conn);
+			}
+			conn->fe_msg->max_fields *= 2;
+		}
+
+		conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+		conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+		conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+		conn->fe_msg->num_fields++;
+	}
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+#define FORMATTED_TS_LEN 128
+	strftime(timestr, FORMATTED_TS_LEN,
+				"%Y-%m-%d %H:%M:%S",
+				localtime(&stamp_time));
+	/* append milliseconds */
+	sprintf(timestr + strlen(timestr), ".%03d", (int) (tval.tv_usec / 1000));
+
+	return timestr;
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn, int msgLen)
+{
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+	}
+	else
+		fprintf(conn->Pfdebug, ">\t%s\t%d",
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqLogMessageString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMessagenchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ */
+static void
+pqLogMessageByte1(PGconn *conn, char v)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_FIRST_BYTE:
+				if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				{
+					char		timestr[128];
+
+					fprintf(conn->Pfdebug, "%s\t<\t",
+							pqLogFormatTimestamp(timestr));
+				}
+				else
+					fprintf(conn->Pfdebug, "<\t");
+
+				/*
+				 * True type of message tagged '\0' is known when next 4 bytes
+				 * is checked. So we delay printing message type to
+				 * pqLogMessageInt()
+				 */
+				if (v != '\0')
+					fprintf(conn->Pfdebug, "%s\t",
+							pqGetProtocolMsgType((unsigned char) v,
+												 MSGDIR_FROM_BACKEND));
+				/* Next, log the message length */
+				conn->be_msg->state = LOG_LENGTH;
+				conn->be_msg->command = v;
+				break;
+
+			case LOG_CONTENTS:
+
+				/*
+				 * Show non-printable data in hex format, including the
+				 * terminating \0 that completes ErrorResponse and
+				 * NoticeResponse messages.
+				 */
+				if (!isprint(v))
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				else
+					fprintf(conn->Pfdebug, "%c", v);
+				pqTraceMaybeBreakLine(1, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+				break;
+		}
+	}
+}
+
+/*
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+static void
+pqLogMessageInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+	{
+		switch (conn->be_msg->state)
+		{
+			case LOG_LENGTH:
+				if (conn->be_msg->command == '\0')
+				{
+					char	   *message_type;
+
+					/*
+					 * We delayed printing message type for special messages;
+					 * they are complete now, so print them.
+					 */
+					if (conn->fe_msg->num_fields > 0)
+					{
+						int			message_addr;
+						uint32		result32;
+						int			result;
+
+						message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+						memcpy(&result32, conn->outBuffer + message_addr, 4);
+						result = (int) pg_ntoh32(result32);
+
+						if (result == NEGOTIATE_SSL_CODE)
+							message_type = "SSLRequest";
+						else if (result == NEGOTIATE_GSS_CODE)
+							message_type = "GSSRequest";
+						else
+							message_type = "StartupMessage";
+					}
+					else
+						message_type = "UnknownMessage";
+					fprintf(conn->Pfdebug, "%s ", message_type);
+				}
+				fprintf(conn->Pfdebug, "%d", v);
+				conn->be_msg->length = v - length;
+				/* Next, log the message contents */
+				conn->be_msg->state = LOG_CONTENTS;
+				pqTraceMaybeBreakLine(0, conn);
+				break;
+
+			case LOG_CONTENTS:
+				fprintf(conn->Pfdebug, "%s%d", prefix, v);
+				pqTraceMaybeBreakLine(length, conn);
+				break;
+
+			default:
+				pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+				break;
+		}
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+static void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,11 +1619,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		/* XXX comment */
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d69d2..d1a74f153b 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +288,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +365,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +377,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +407,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +471,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1640,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index c266ad5b13..ad04b65513 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,8 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
-extern void PQtrace(PGconn *conn, FILE *debug_port);
+#define PQTRACE_SUPPRESS_TIMESTAMPS		(1 << 0)
+extern void PQtrace(PGconn *conn, FILE *debug_port, bits32 flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db498369c..1f32af8119 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -375,6 +383,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	bits32		traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -672,6 +685,9 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn, bits32 flags);
+extern void pqTraceUninit(PGconn *conn);
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 943142ced8..bbf5022513 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1522,6 +1522,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1537,6 +1538,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3271,6 +3274,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#79tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Alvaro Herrera (#78)
RE: libpq debug log

Hello Alvaro-san, Iwata-san,

First of all, thank you Alvaro-san really a lot for your great help. I'm glad you didn't lose interest and time for this patch yet. (Iwata-san is my colleague.)

From: Alvaro Herrera <alvherre@alvh.no-ip.org>

That's true, but it'd require that we move PQtrace() to fe-misc.c, because
pqTraceInit() uses definitions that are private to that file.

Ah, that was the reason for separation. Then, I'm fine with either keeping the separation, or integrating the two functions in fe-misc.c with PQuntrace() being also there. I kind of think the latter is better, because of code readability and, because tracing facility is not a connection-related reature so categorizing it as "misc" feels natural.

What's this code for? I think it's sufficient that PQuntrace() frees b_msg

and PQtrace() always allocates b_msg.

The point is to be able to cope with a connection that calls PQtrace() multiple
times, with or without an intervening PQuntrace().
We'd make no friends if we made PQtrace() crash, or leak memory if called a
second time ...

HEAD's PQtrace() always call PQuntrace() and then re-initialize from scratch. So, if we keep that flow, the user can call PQtrace() multiple in a row.

I think trying to apply tabs inside the message contents is going to be more
confusing than helpful.

Agreed.

Plus, you may as well print the invalid message type. Why don't you do

something like this:

+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource) {
+	static char unknown_message[64];
+	char *msg = NULL;

+ if (commsource == FROM_BACKEND && c <

lengthof(protocol_message_type_b))

+		msg = protocol_message_type_b[c];
+	else if (commsource == FROM_FRONTEND && c <

lengthof(protocol_message_type_f))

+		msg = protocol_message_type_f[c];
+
+	if (msg == NULL)
+	{
+		sprintf(unknown_message, "Unknown message %02x", c);
+		msg = unknown_message;
+	}
+
+	return msg;
+}

Right. I had written something like this, but in the end decided not to bother
because it doesn't seem terribly important. You can't do exactly what you
suggest, because it has the problem that you're returning a stack-allocated
variable even though your stack is about to be blown away. My copy had a
static buffer that was malloc()ed on first use (and if allocation fails, return a
const string). Anyway, I fixed the crasher problem.

My suggestion included static qualifier to not use the stack, but it doesn't work anyway in multi-threaded applications. So I agree that we don't print the invalid message type value.

(18)
if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);

To match the style for backend messages with pqLogMsgByte1 etc., why

don't you wrap the conn->Pfdebug check in the macro?

I think these macros are useless and should be removed. There's more
verbosity and confusion caused by them, than the clarity they provide.

Agreed.

This patch needs more work still, of course.

Yes, she is updating the patch based on the feedback from you, Kirk-san, and me.

By the way, I just looked at the beginning of v12.

-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);

Can we change the signature of an API function?

Iwata-san,
Please integrate Alvaro-san's patch with yours. Next week is the last week in this CF, so do your best to post the patch by next Monday or so (before Alvaro-san loses interest or time.) Then I'll review it ASAP.

Regards
Takayuki Tsunakawa

#80'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#79)
Re: libpq debug log

Hello, just two quick comments on this,

On 2021-Jan-22, tsunakawa.takay@fujitsu.com wrote:

From: Alvaro Herrera <alvherre@alvh.no-ip.org>

That's true, but it'd require that we move PQtrace() to fe-misc.c, because
pqTraceInit() uses definitions that are private to that file.

Ah, that was the reason for separation. Then, I'm fine with either
keeping the separation, or integrating the two functions in fe-misc.c
with PQuntrace() being also there. I kind of think the latter is
better, because of code readability and, because tracing facility is
not a connection-related reature so categorizing it as "misc" feels
natural.

Maybe we can create a new file specifically for this to avoid mixing
unrelated stuff in fe-misc.c -- say fe-trace.c where all this new
tracing stuff goes.

The point is to be able to cope with a connection that calls PQtrace() multiple
times, with or without an intervening PQuntrace().
We'd make no friends if we made PQtrace() crash, or leak memory if called a
second time ...

HEAD's PQtrace() always call PQuntrace() and then re-initialize from
scratch. So, if we keep that flow, the user can call PQtrace()
multiple in a row.

Oh, of course.

--
�lvaro Herrera Valdivia, Chile
"No necesitamos banderas
No reconocemos fronteras" (Jorge Gonz�lez)

#81tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#79)
RE: libpq debug log

First, some possibly major questions:

(23)
From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>

Maybe we can create a new file specifically for this to avoid mixing
unrelated stuff in fe-misc.c -- say fe-trace.c where all this new
tracing stuff goes.

What do you think about this suggestion? I think this is reasonable, including moving PQtrace/PQuntrace to the new file.

OTOH, fe-misc also has humble reasons to contain them. One is that the file header comment is as follows, accepting miscellaneous stuff. Another is that although most functions in fe-misc.c are related to protocol data transmission and receipt, several functions at the end of the file are already not related.

* fe-misc.c
*
* DESCRIPTION
* miscellaneous useful functions

(24)
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);
 </synopsis>
      </para>
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>(flags & TRACE_SUPPRESS_TIMESTAMPS)</literal> is
+      true, then timestamps are not printed with each message.
+     </para>

As I asked in the previous mail, I'm afraid we cannot change the signature of existing API functions. If we want this flag bits, we have to add something like PQtraceEx(), don't we?

The flag name differs between in the manual and in the source code:

+#define PQTRACE_SUPPRESS_TIMESTAMPS (1 << 0)

P.S.
Also, please note this as:

Also, why don't you try running the regression tests with a temporary modification to PQtrace() to output the trace to a file? The sole purpose is to confirm that this patch doesn't make the test crash (core dump).

Regards
Takayuki Tsunakawa

#82iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: 'Alvaro Herrera' (#80)
1 attachment(s)
RE: libpq debug log

Hello, Alvaro san , Tsunakawa san

Thank you for your help. It was very helpful.

Please integrate Alvaro-san's patch with yours. Next week is the last week in this CF, so do your best to post the patch by next Monday or so (before Alvaro-san loses interest or time.) Then I'll review it ASAP.

I integrated my fix and update v12 patch. All previous review comments apply to this patch.

Regards,
Aya Iwata
Fujitsu

Attachments:

v13-0001-libpq-trace.patchapplication/octet-stream; name=v13-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b7a82453f0..b54af2cf14 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,19 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);
 </synopsis>
      </para>
 
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>(flags & TRACE_SUPPRESS_TIMESTAMPS)</literal> is
+      true, then timestamps are not printed with each message.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583aa9..5bc0a610c0 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6761,12 +6761,28 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 }
 
 void
-PQtrace(PGconn *conn, FILE *debug_port)
+PQtrace(PGconn *conn, FILE *debug_port, bits32 flags)
 {
 	if (conn == NULL)
 		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (!debug_port)
+		return;
+	if (pqTraceInit(conn, flags))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
 }
 
 void
@@ -6779,6 +6795,8 @@ PQuntrace(PGconn *conn)
 		fflush(conn->Pfdebug);
 		conn->Pfdebug = NULL;
 	}
+	pqTraceUninit(conn);
+	conn->traceFlags = 0;
 }
 
 PQnoticeReceiver
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6acd89..cf2484f975 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -51,13 +51,168 @@
 #include "libpq-int.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
+#include "pgtime.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqStoreFeMsgStart(PGconn *conn, char type);
+static void pqLogFrontendMsg(PGconn *conn, int msgLen);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogMessageByte1(PGconn *conn, char v);
+static void pqLogMessageInt(PGconn *conn, int v, int length);
+static void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,7 +240,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMessageByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +256,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +294,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMessageString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -157,18 +311,19 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 	return pqGets_internal(buf, conn, false);
 }
 
-
 /*
  * pqPuts: write a null-terminated string to the current message
  */
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +344,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMessagenchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +364,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqLogMessagenchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +382,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +422,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMessageInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +437,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +461,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -473,8 +619,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 	}
 
 	/* realloc failed. Probably out of memory */
-	appendPQExpBufferStr(&conn->errorMessage,
-						 "cannot allocate memory for input buffer\n");
+	appendPQExpBuffer(&conn->errorMessage,
+					  "cannot allocate memory for input buffer\n");
 	return EOF;
 }
 
@@ -535,8 +681,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqStoreFeMsgStart(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +717,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +744,462 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+bool
+pqTraceInit(PGconn *conn, bits32 flags)
+{
+	conn->traceFlags = flags;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(sizeof(offsetof(pqFrontendMessage, fields)) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+
+	return true;
+}
+
+/*
+ * Deallocate frontend-message tracking memory.  We only do this because
+ * it can grow arbitrarily large, and skip it in the initial state, because
+ * it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		pfree(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+		conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX / 2)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					sizeof(pqFrontendMessage) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+#define FORMATTED_TS_LEN 128
+	strftime(timestr, FORMATTED_TS_LEN,
+				"%Y-%m-%d %H:%M:%S",
+				localtime(&stamp_time));
+	/* append milliseconds */
+	sprintf(timestr + strlen(timestr), ".%03d", (int) (tval.tv_usec / 1000));
+
+	return timestr;
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn, int msgLen)
+{
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+	}
+	else
+		fprintf(conn->Pfdebug, ">\t%s\t%d",
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqLogMessageString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMessagenchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ */
+static void
+pqLogMessageByte1(PGconn *conn, char v)
+{
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+			{
+				char		timestr[128];
+
+				fprintf(conn->Pfdebug, "%s\t<\t",
+						pqLogFormatTimestamp(timestr));
+			}
+			else
+				fprintf(conn->Pfdebug, "<\t");
+
+			/*
+				* True type of message tagged '\0' is known when next 4 bytes
+				* is checked. So we delay printing message type to
+				* pqLogMessageInt()
+				*/
+			if (v != '\0')
+				fprintf(conn->Pfdebug, "%s\t",
+						pqGetProtocolMsgType((unsigned char) v,
+												MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+static void
+pqLogMessageInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			if (conn->be_msg->command == '\0')
+			{
+				char	   *message_type;
+
+				/*
+					* We delayed printing message type for special messages;
+					* they are complete now, so print them.
+					*/
+				if (conn->fe_msg->num_fields > 0)
+				{
+					int			message_addr;
+					uint32		result32;
+					int			result;
+
+					message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else if (result == NEGOTIATE_GSS_CODE)
+						message_type = "GSSRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownMessage";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+			}
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+static void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,11 +1611,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		/* XXX comment */
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d69d2..d1a74f153b 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +288,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +365,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +377,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +407,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +471,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1640,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index c266ad5b13..ad04b65513 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,8 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
-extern void PQtrace(PGconn *conn, FILE *debug_port);
+#define PQTRACE_SUPPRESS_TIMESTAMPS		(1 << 0)
+extern void PQtrace(PGconn *conn, FILE *debug_port, bits32 flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db498369c..1f32af8119 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -375,6 +383,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	bits32		traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -672,6 +685,9 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn, bits32 flags);
+extern void pqTraceUninit(PGconn *conn);
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 721b230bf2..18475a0177 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1522,6 +1522,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1537,6 +1538,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3272,6 +3275,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#83tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#79)
1 attachment(s)
RE: libpq debug log

Iwata-san, all,

Strangely, Iwata-san's latest mail she sent today at 10:34 JST hasn't appeared on pgsql-hackers yet after more than 6 hours. It is not reflected in the CF entry [1]https://commitfest.postgresql.org/31/2889/. So, I'm putting her original mail below. The v13 patch attached to the original mail is attached to this mail.

I guess only Alvaro-san and me, who are listed in To:, received the original mail.

Iwata-san,
Please ask pgsql-www list about this problem.

[1]: https://commitfest.postgresql.org/31/2889/
https://commitfest.postgresql.org/31/2889/

Regards
Takayuki Tsunakawa

Show quoted text

-----Original Message-----
From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>
Sent: Monday, January 25, 2021 10:34 AM
To: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>; Tsunakawa, Takayuki/綱川 貴
之 <tsunakawa.takay@fujitsu.com>
Cc: 'pgsql-hackers@lists.postgresql.org'
<pgsql-hackers@lists.postgresql.org>
Subject: RE: libpq debug log

Hello, Alvaro san , Tsunakawa san

Thank you for your help. It was very helpful.

Please integrate Alvaro-san's patch with yours. Next week is the last week

in this CF, so do your best to post the patch by next Monday or so (before
Alvaro-san loses interest or time.) Then I'll review it ASAP.
I integrated my fix and update v12 patch. All previous review comments apply
to this patch.

Regards,
Aya Iwata
Fujitsu

Attachments:

v13-0001-libpq-trace.patchapplication/octet-stream; name=v13-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b7a82453f0..b54af2cf14 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,19 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);
 </synopsis>
      </para>
 
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>(flags & TRACE_SUPPRESS_TIMESTAMPS)</literal> is
+      true, then timestamps are not printed with each message.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583aa9..5bc0a610c0 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6761,12 +6761,28 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 }
 
 void
-PQtrace(PGconn *conn, FILE *debug_port)
+PQtrace(PGconn *conn, FILE *debug_port, bits32 flags)
 {
 	if (conn == NULL)
 		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (!debug_port)
+		return;
+	if (pqTraceInit(conn, flags))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
 }
 
 void
@@ -6779,6 +6795,8 @@ PQuntrace(PGconn *conn)
 		fflush(conn->Pfdebug);
 		conn->Pfdebug = NULL;
 	}
+	pqTraceUninit(conn);
+	conn->traceFlags = 0;
 }
 
 PQnoticeReceiver
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6acd89..cf2484f975 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -51,13 +51,168 @@
 #include "libpq-int.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
+#include "pgtime.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqStoreFeMsgStart(PGconn *conn, char type);
+static void pqLogFrontendMsg(PGconn *conn, int msgLen);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogMessageByte1(PGconn *conn, char v);
+static void pqLogMessageInt(PGconn *conn, int v, int length);
+static void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,7 +240,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMessageByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +256,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +294,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMessageString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -157,18 +311,19 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 	return pqGets_internal(buf, conn, false);
 }
 
-
 /*
  * pqPuts: write a null-terminated string to the current message
  */
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +344,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMessagenchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +364,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqLogMessagenchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +382,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +422,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMessageInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +437,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +461,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -473,8 +619,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 	}
 
 	/* realloc failed. Probably out of memory */
-	appendPQExpBufferStr(&conn->errorMessage,
-						 "cannot allocate memory for input buffer\n");
+	appendPQExpBuffer(&conn->errorMessage,
+					  "cannot allocate memory for input buffer\n");
 	return EOF;
 }
 
@@ -535,8 +681,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqStoreFeMsgStart(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +717,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +744,462 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+bool
+pqTraceInit(PGconn *conn, bits32 flags)
+{
+	conn->traceFlags = flags;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(sizeof(offsetof(pqFrontendMessage, fields)) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+
+	return true;
+}
+
+/*
+ * Deallocate frontend-message tracking memory.  We only do this because
+ * it can grow arbitrarily large, and skip it in the initial state, because
+ * it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		pfree(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+		conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX / 2)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					sizeof(pqFrontendMessage) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+#define FORMATTED_TS_LEN 128
+	strftime(timestr, FORMATTED_TS_LEN,
+				"%Y-%m-%d %H:%M:%S",
+				localtime(&stamp_time));
+	/* append milliseconds */
+	sprintf(timestr + strlen(timestr), ".%03d", (int) (tval.tv_usec / 1000));
+
+	return timestr;
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn, int msgLen)
+{
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+	}
+	else
+		fprintf(conn->Pfdebug, ">\t%s\t%d",
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqLogMessageString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMessagenchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ */
+static void
+pqLogMessageByte1(PGconn *conn, char v)
+{
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+			{
+				char		timestr[128];
+
+				fprintf(conn->Pfdebug, "%s\t<\t",
+						pqLogFormatTimestamp(timestr));
+			}
+			else
+				fprintf(conn->Pfdebug, "<\t");
+
+			/*
+				* True type of message tagged '\0' is known when next 4 bytes
+				* is checked. So we delay printing message type to
+				* pqLogMessageInt()
+				*/
+			if (v != '\0')
+				fprintf(conn->Pfdebug, "%s\t",
+						pqGetProtocolMsgType((unsigned char) v,
+												MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+static void
+pqLogMessageInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			if (conn->be_msg->command == '\0')
+			{
+				char	   *message_type;
+
+				/*
+					* We delayed printing message type for special messages;
+					* they are complete now, so print them.
+					*/
+				if (conn->fe_msg->num_fields > 0)
+				{
+					int			message_addr;
+					uint32		result32;
+					int			result;
+
+					message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else if (result == NEGOTIATE_GSS_CODE)
+						message_type = "GSSRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownMessage";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+			}
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+static void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,11 +1611,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		/* XXX comment */
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d69d2..d1a74f153b 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +288,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +365,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +377,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +407,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +471,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1640,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index c266ad5b13..ad04b65513 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,8 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
-extern void PQtrace(PGconn *conn, FILE *debug_port);
+#define PQTRACE_SUPPRESS_TIMESTAMPS		(1 << 0)
+extern void PQtrace(PGconn *conn, FILE *debug_port, bits32 flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db498369c..1f32af8119 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -375,6 +383,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	bits32		traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -672,6 +685,9 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn, bits32 flags);
+extern void pqTraceUninit(PGconn *conn);
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 721b230bf2..18475a0177 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1522,6 +1522,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1537,6 +1538,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3272,6 +3275,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#84k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#82)
RE: libpq debug log

Hello,

I have not tested nor review the latest patch changes yet, but I am reporting the compiler errors.
I am trying to compile the patch since V12 (Alvaro's version), but the following needs to
be fixed too because of the complaints: doc changes and undeclared INT_MAX

libpq.sgml:5893: parser error : xmlParseEntityRef: no name
of tracing. If <literal>(flags & TRACE_SUPPRESS_TIMESTAMPS)</literal> is
^
libpq.sgml:8799: parser error : chunk is not well balanced
postgres.sgml:195: parser error : Failure to process entity libpq
&libpq;
^
postgres.sgml:195: parser error : Entity 'libpq' not defined
&libpq;
^
...
fe-misc.c: In function ‘pqStoreFrontendMsg’:
fe-misc.c:903:35: error: ‘INT_MAX’ undeclared (first use in this function)
if (conn->fe_msg->max_fields > INT_MAX / 2)

Regards,
Kirk Jamison

#85iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#83)
RE: libpq debug log

Tsunakawa san,

Strangely, Iwata-san's latest mail she sent today at 10:34 JST hasn't appeared
on pgsql-hackers yet after more than 6 hours. It is not reflected in the CF
entry [1]. So, I'm putting her original mail below. The v13 patch attached to
the original mail is attached to this mail.

I solved this problem. Thank you for sharing my email.

Regards,
Aya Iwata
Fujitsu

#86tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#82)
RE: libpq debug log

Iwata-san,

(25)
+		conn->fe_msg = malloc(sizeof(offsetof(pqFrontendMessage, fields)) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));

It's incorrect that offsetof() is nested in sizeof(). See my original review comment.

(26)
+bool
+pqTraceInit(PGconn *conn, bits32 flags)
+{
...
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+	}
...
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;

The former is redundant?

(27)
+/*
+ * Deallocate frontend-message tracking memory.  We only do this because
+ * it can grow arbitrarily large, and skip it in the initial state, because
+ * it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		pfree(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+}

I thought this function plays the reverse role of pqTraceInit(), but it only frees memory for frontend messages. Plus, use free() instead of pfree(), because malloc() is used to allocate memory.

(28)
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);

bits32 can't be used because it's a data type defined in c.h, which should not be exposed to applications. I think you can just int or something.

Considering these and the compilation error Kirk-san found, I'd like you to do more self-review before I resume this review.

Regards
Takayuki Tsunakawa

#87'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#86)
Re: libpq debug log

On 2021-Jan-25, tsunakawa.takay@fujitsu.com wrote:

Iwata-san,

[...]

Considering these and the compilation error Kirk-san found, I'd like
you to do more self-review before I resume this review.

Kindly note that these errors are all mine.

Thanks for the review

--
�lvaro Herrera 39�49'30"S 73�17'W
#error "Operator lives in the wrong universe"
("Use of cookies in real-time system development", M. Gleixner, M. Mc Guire)

#88k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: 'Alvaro Herrera' (#87)
1 attachment(s)
RE: libpq debug log

On Mon, Jan 25, 2021 10:11 PM (JST), Alvaro Herrera wrote:

On 2021-Jan-25, tsunakawa.takay@fujitsu.com wrote:

Iwata-san,

[...]

Considering these and the compilation error Kirk-san found, I'd like
you to do more self-review before I resume this review.

Kindly note that these errors are all mine.

Thanks for the review

Hello,
To make the CFBot happy, I fixed the compiler errors.
I have not included here the change proposal to move the tracing functions
to a new file: fe-trace.c or something, so I retained the changes .
Maybe Iwata-san can consider the proposal.
Currently, both pqTraceInit() and pqTraceUninit() are called only by one function.

Summary:
1. I changed the bits32 flags to int flags
2. I assumed INT_MAX value is the former MAX_FRONTEND_MSGS
I defined it to solve the compile error. Please tell if the value was wrong.
3. Fixed the doc errors
4. Added freeing of be_msg in pqTraceUninit()
5. Addressed the following comments:

(25)
+ conn->fe_msg =
malloc(sizeof(offsetof(pqFrontendMessage, fields)) +
+
DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));

It's incorrect that offsetof() is nested in sizeof(). See my original
review comment.

I removed the initial sizeof().

(26)
+bool
+pqTraceInit(PGconn *conn, bits32 flags) {
...
+		conn->be_msg->state = LOG_FIRST_BYTE;
+		conn->be_msg->length = 0;
+	}
...
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;

The former is redundant?

Right. I've removed the former.

(27)
+/*
+ * Deallocate frontend-message tracking memory.  We only do this 
+because
+ * it can grow arbitrarily large, and skip it in the initial state, 
+because
+ * it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		pfree(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+}

I thought this function plays the reverse role of pqTraceInit(), but
it only frees memory for frontend messages. Plus, use free() instead
of pfree(), because
malloc() is used to allocate memory.

Fixed to use free(). Also added the freeing of be_msg.

(28)
+void PQtrace(PGconn *conn, FILE *stream, bits32 flags);

bits32 can't be used because it's a data type defined in c.h, which
should not be exposed to applications. I think you can just int or something.

I used int. It still works to suppress the timestamp when flags is true.

In my environment, when flags is false(0):
2021-01-28 06:26:11.368 > Query 27 "COPY country TO STDOUT"
2021-01-28 06:26:11.368 > Query -1
2021-01-28 06:26:11.369 < CopyOutResponse 13 \x00 #3 #0 #0 #0
2021-01-28 06:26:11.369 < CopyDone 4

2021-01-28 06:26:11.369 < CopyDone 4
2021-01-28 06:26:11.369 < CopyDone 4
2021-01-28 06:26:11.369 < CommandComplete 11 "COPY 0"
2021-01-28 06:26:11.369 < ReadyForQuery 5
2021-01-28 06:26:11.369 > Terminate 4
2021-01-28 06:26:11.369 > Terminate -1

When flags is true(1), running the same query:

Query 27 "COPY country TO STDOUT"
Query -1

< CopyOutResponse 13 \x00 #3 #0 #0 #0
< CopyDone 4

< CopyDone 4
< CopyDone 4
< CommandComplete 11 "COPY 0"
< ReadyForQuery 5

Terminate 4
Terminate -1

Regards,
Kirk Jamison

Attachments:

v14-0001-libpq-trace.patchapplication/octet-stream; name=v14-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b7a8245..333dc3d 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,19 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, int flags);
 </synopsis>
      </para>
 
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> &amp; <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>) is
+      true, then timestamps are not printed with each message.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583..f1d79b9 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6761,12 +6761,28 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 }
 
 void
-PQtrace(PGconn *conn, FILE *debug_port)
+PQtrace(PGconn *conn, FILE *debug_port, int flags)
 {
 	if (conn == NULL)
 		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+
 	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
+	if (!debug_port)
+		return;
+	if (pqTraceInit(conn, flags))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
 }
 
 void
@@ -6779,6 +6795,8 @@ PQuntrace(PGconn *conn)
 		fflush(conn->Pfdebug);
 		conn->Pfdebug = NULL;
 	}
+	pqTraceUninit(conn);
+	conn->traceFlags = 0;
 }
 
 PQnoticeReceiver
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..318ef4a 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -51,13 +51,169 @@
 #include "libpq-int.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
+#include "pgtime.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+#define INT_MAX	(2048/2)		/* maximum array size */
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqStoreFeMsgStart(PGconn *conn, char type);
+static void pqLogFrontendMsg(PGconn *conn, int msgLen);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogMessageByte1(PGconn *conn, char v);
+static void pqLogMessageInt(PGconn *conn, int v, int length);
+static void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
 
 /*
  * PQlibVersion: return the libpq version number
@@ -85,7 +241,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMessageByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +257,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +295,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMessageString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -157,18 +312,19 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 	return pqGets_internal(buf, conn, false);
 }
 
-
 /*
  * pqPuts: write a null-terminated string to the current message
  */
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +345,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMessagenchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +365,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqLogMessagenchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +383,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +423,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMessageInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +438,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +462,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -473,8 +620,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 	}
 
 	/* realloc failed. Probably out of memory */
-	appendPQExpBufferStr(&conn->errorMessage,
-						 "cannot allocate memory for input buffer\n");
+	appendPQExpBuffer(&conn->errorMessage,
+					  "cannot allocate memory for input buffer\n");
 	return EOF;
 }
 
@@ -535,8 +682,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqStoreFeMsgStart(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +718,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +745,465 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+bool
+pqTraceInit(PGconn *conn, int flags)
+{
+	conn->traceFlags = flags;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+
+	return true;
+}
+
+/*
+ * Deallocate FE/BE message tracking memory.  We only do this because
+ * FE message can grow arbitrarily large, and skip it in the initial state,
+ * because it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+		conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					sizeof(pqFrontendMessage) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+#define FORMATTED_TS_LEN 128
+	strftime(timestr, FORMATTED_TS_LEN,
+				"%Y-%m-%d %H:%M:%S",
+				localtime(&stamp_time));
+	/* append milliseconds */
+	sprintf(timestr + strlen(timestr), ".%03d", (int) (tval.tv_usec / 1000));
+
+	return timestr;
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn, int msgLen)
+{
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+	}
+	else
+		fprintf(conn->Pfdebug, ">\t%s\t%d",
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqLogMessageString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMessagenchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ */
+static void
+pqLogMessageByte1(PGconn *conn, char v)
+{
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+			{
+				char		timestr[128];
+
+				fprintf(conn->Pfdebug, "%s\t<\t",
+						pqLogFormatTimestamp(timestr));
+			}
+			else
+				fprintf(conn->Pfdebug, "<\t");
+
+			/*
+				* True type of message tagged '\0' is known when next 4 bytes
+				* is checked. So we delay printing message type to
+				* pqLogMessageInt()
+				*/
+			if (v != '\0')
+				fprintf(conn->Pfdebug, "%s\t",
+						pqGetProtocolMsgType((unsigned char) v,
+												MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+static void
+pqLogMessageInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			if (conn->be_msg->command == '\0')
+			{
+				char	   *message_type;
+
+				/*
+					* We delayed printing message type for special messages;
+					* they are complete now, so print them.
+					*/
+				if (conn->fe_msg->num_fields > 0)
+				{
+					int			message_addr;
+					uint32		result32;
+					int			result;
+
+					message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else if (result == NEGOTIATE_GSS_CODE)
+						message_type = "GSSRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownMessage";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+			}
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+static void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,11 +1615,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		/* XXX comment */
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..d1a74f1 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +288,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +365,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +377,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +407,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +471,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1640,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index c266ad5..4f3654c 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,8 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
-extern void PQtrace(PGconn *conn, FILE *debug_port);
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
+extern void PQtrace(PGconn *conn, FILE *debug_port, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db4983..eed4ac7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -375,6 +383,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -672,6 +685,9 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn, int flags);
+extern void pqTraceUninit(PGconn *conn);
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 721b230..18475a0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1522,6 +1522,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1537,6 +1538,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3272,6 +3275,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#89k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#81)
RE: libpq debug log

On Mon, January 25, 2021 4:13 PM (JST), Tsunakawa-san wrote:.

Also, please note this as:

Also, why don't you try running the regression tests with a temporary

modification to PQtrace() to output the trace to a file? The sole purpose is
to confirm that this patch doesn't make the test crash (core dump).

I realized that existing libpq regression tests in src/test/examples do not
test PQtrace(). At the same time, no other functions Is calling PQtrace(),
so it's expected that by default if there are no compilation errors, then it
will pass all the tests. And it did.

So to check that the tracing feature is enabled and, as requested, outputs
a trace file, I temporarily modified a bit of testlibpq.c instead **based from
my current environment settings**, and ran the regression test.

diff --git a/src/test/examples/testlibpq.c b/src/test/examples/testlibpq.c
index 0372781..ae163e4 100644
--- a/src/test/examples/testlibpq.c
+++ b/src/test/examples/testlibpq.c
@@ -26,6 +26,7 @@ main(int argc, char **argv)
        int                     nFields;
        int                     i,
                                j;
+       FILE            *trace_file;

/*
* If the user supplies a parameter on the command line, use it as the
@@ -40,6 +41,9 @@ main(int argc, char **argv)
/* Make a connection to the database */
conn = PQconnectdb(conninfo);

+       trace_file = fopen("/home/postgres/tracelog/trace.out","w");
+       PQtrace(conn, trace_file, 0);
+
        /* Check to see that the backend connection was successfully made */
        if (PQstatus(conn) != CONNECTION_OK)
        {
@@ -127,5 +131,6 @@ main(int argc, char **argv)
        /* close the connection to the database and cleanup */
        PQfinish(conn);

+ fclose(trace_file);
return 0;
}

make -C src/test/examples/ PROGS=testlibpq
./src/test/examples/testlibpq

I've verified that it works (no crash) and outputs a trace file to the PATH that I set in libpqtest.

The trace file contains the following log for the testlibq:

2021-01-28 09:22:28.155 > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"
2021-01-28 09:22:28.156 > Query -1
2021-01-28 09:22:28.157 < RowDescription 35 #1 "set_config" 0 #0 25 #65535 -1 #0
2021-01-28 09:22:28.157 < DataRow 10 #1 0
2021-01-28 09:22:28.157 < CommandComplete 13 "SELECT 1"
2021-01-28 09:22:28.157 < ReadyForQuery 5
2021-01-28 09:22:28.157 < ReadyForQuery 5 I
2021-01-28 09:22:28.157 > Query 10 "BEGIN"
2021-01-28 09:22:28.157 > Query -1
2021-01-28 09:22:28.157 < CommandComplete 10 "BEGIN"
2021-01-28 09:22:28.157 < ReadyForQuery 5
2021-01-28 09:22:28.157 < ReadyForQuery 5 T
2021-01-28 09:22:28.157 > Query 58 "DECLARE myportal CURSOR FOR select * from pg_database"
2021-01-28 09:22:28.157 > Query -1
2021-01-28 09:22:28.159 < CommandComplete 19 "DECLARE CURSOR"
2021-01-28 09:22:28.159 < ReadyForQuery 5
2021-01-28 09:22:28.159 < ReadyForQuery 5 T
2021-01-28 09:22:28.159 > Query 26 "FETCH ALL in myportal"
2021-01-28 09:22:28.159 > Query -1
2021-01-28 09:22:28.159 < RowDescription 405 #14 "oid" 1262 #1 26 #4 -1 #0 "datname" 1262 #2 19 #64 -1 #0 "datdba" 1262 #3 26 #4 -1 #0 "encoding" 1262 #4 23 #4 -1 #0 "datcollate" 1262 #5 19 #64 -1 #0 "datctype" 1262 #6 19 #64 -1 #0 "datistemplate" 1262 #7 16 #1 -1 #0 "datallowconn" 1262 #8 16 #1 -1 #0 "datconnlimit" 1262 #9 23 #4 -1 #0 "datlastsysoid" 1262 #10 26 #4 -1 #0 "datfrozenxid" 1262 #11 28 #4 -1 #0 "datminmxid" 1262 #12 28 #4 -1 #0 "dattablespace" 1262 #13 26 #4 -1 #0 "datacl" 1262 #14 1034 #65535 -1 #0
2021-01-28 09:22:28.159 < DataRow 117 #14 5 '13756' 8 'postgres' 2 '10' 1 '6' 11 'en_US.UTF-8' 11 'en_US.UTF-8' 1 'f' 1 't' 2 '-1' 5 '13755' 3 '502' 1 '1' 4 '1663' -1
2021-01-28 09:22:28.159 < DataRow 149 #14 1 '1' 9 'template1' 2 '10' 1 '6' 11 'en_US.UTF-8' 11 'en_US.UTF-8' 1 't' 1 't' 2 '-1' 5 '13755' 3 '502' 1 '1' 4 '1663' 35 '{=c/postgres,postgres=CTc/postgres}'
2021-01-28 09:22:28.159 < DataRow 153 #14 5 '13755' 9 'template0' 2 '10' 1 '6' 11 'en_US.UTF-8' 11 'en_US.UTF-8' 1 't' 1 'f' 2 '-1' 5 '13755' 3 '502' 1 '1' 4 '1663' 35 '{=c/postgres,postgres=CTc/postgres}'
2021-01-28 09:22:28.159 < CommandComplete 12 "FETCH 3"
2021-01-28 09:22:28.159 < ReadyForQuery 5
2021-01-28 09:22:28.159 < ReadyForQuery 5 T
2021-01-28 09:22:28.159 > Query 19 "CLOSE myportal"
2021-01-28 09:22:28.159 > Query -1
2021-01-28 09:22:28.159 < CommandComplete 17 "CLOSE CURSOR"
2021-01-28 09:22:28.159 < ReadyForQuery 5
2021-01-28 09:22:28.159 < ReadyForQuery 5 T
2021-01-28 09:22:28.159 > Query 8 "END"
2021-01-28 09:22:28.159 > Query -1
2021-01-28 09:22:28.160 < CommandComplete 11 "COMMIT"
2021-01-28 09:22:28.160 < ReadyForQuery 5
2021-01-28 09:22:28.160 < ReadyForQuery 5 I
2021-01-28 09:22:28.160 > Terminate 4
2021-01-28 09:22:28.160 > Terminate -1

Regards,
Kirk Jamison

#90'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: k.jamison@fujitsu.com (#89)
Re: libpq debug log

On 2021-Jan-28, k.jamison@fujitsu.com wrote:

I realized that existing libpq regression tests in src/test/examples do not
test PQtrace(). At the same time, no other functions Is calling PQtrace(),
so it's expected that by default if there are no compilation errors, then it
will pass all the tests. And it did.

So to check that the tracing feature is enabled and, as requested, outputs
a trace file, I temporarily modified a bit of testlibpq.c instead **based from
my current environment settings**, and ran the regression test.

Yeah. It seems useful to build a real test harness based on the new
tracing functionality. And that is precisely why I added the option to
suppress timestamps: so that we can write trace files that do not vary
from run to run. That allows us to have an "expected" trace to which we
can compare the trace file produced by the actual run. I had the idea
that instead of a timestamp we would print a message counter. I didn't
implement that, however.

So what we need to do now, before we cast in stone such expected files,
is ensure that the trace file produced by some program (be it
testlibpq.c or some other program) accurately matches what is sent over
the network. If the tracing infrastructure has bugs, then the trace
will contain artifacts, and that's something to avoid. For example, in
the trace you paste, why are there two "Query" messages every time, with
the second one having length -1? I think this is a bug I introduced
recently.

2021-01-28 09:22:28.155 > Query 59 "SELECT pg_catalog.set_config('search_path', '', false)"
2021-01-28 09:22:28.156 > Query -1
2021-01-28 09:22:28.157 < RowDescription 35 #1 "set_config" 0 #0 25 #65535 -1 #0
2021-01-28 09:22:28.157 < DataRow 10 #1 0
2021-01-28 09:22:28.157 < CommandComplete 13 "SELECT 1"

And afterwards, why are always there two ReadyForQuery messages?

2021-01-28 09:22:28.157 < ReadyForQuery 5
2021-01-28 09:22:28.157 < ReadyForQuery 5 I

In ReadyForQuery, we also get a transaction status. In this case it's
"I" which means "Idle" (pretty mysterious if you don't know, as was my
case). A bunch of other values are possible. Do we want to translate
this to human-readable flags, similarly to how we translate message tag
chars to message names? Seems useful to me. Or maybe it's
overengineering, about which see below.

2021-01-28 09:22:28.159 < RowDescription 405 #14 "oid" 1262 #1 26 #4 -1 #0 "datname" 1262 #2 19 #64 -1 #0 "datdba" 1262 #3 26 #4 -1 #0 "encoding" 1262 #4 23 #4 -1 #0 "datcollate" 1262 #5 19 #64 -1 #0 "datctype" 1262 #6 19 #64 -1 #0 "datistemplate" 1262 #7 16 #1 -1 #0 "datallowconn" 1262 #8 16 #1 -1 #0 "datconnlimit" 1262 #9 23 #4 -1 #0 "datlastsysoid" 1262 #10 26 #4 -1 #0 "datfrozenxid" 1262 #11 28 #4 -1 #0 "datminmxid" 1262 #12 28 #4 -1 #0 "dattablespace" 1262 #13 26 #4 -1 #0 "datacl" 1262 #14 1034 #65535 -1 #0

This is terribly unreadable, but at this point that's okay because we're
just doing tracing at a very low level. In the future we could add some
new flag PQTRACE_INTERPRET_MESSAGES or something, which changes the
output format to something more readable. Or maybe nobody cares to
spend time doing that.

Cheers

--
�lvaro Herrera Valdivia, Chile
"Para tener m�s hay que desear menos"

#91tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: k.jamison@fujitsu.com (#88)
RE: libpq debug log

From: Jamison, Kirk/ジャミソン カーク <k.jamison@fujitsu.com>

To make the CFBot happy, I fixed the compiler errors.
I have not included here the change proposal to move the tracing functions to a
new file: fe-trace.c or something, so I retained the changes .
Maybe Iwata-san can consider the proposal.
Currently, both pqTraceInit() and pqTraceUninit() are called only by one
function.

I confirmed all mentioned points have been fixed. Additional comments are:

(29)
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, int flags);

As I said before, I'm afraid we cannot change the signature of existing API functions. Please leave the signature of this function unchanged, and add a new function like PQtraceEx() that adds the flags argument.

I guess the purpose of adding the flag argument is to not output the timestamp by default, because getting timestamps would incur significant overhead (however, I'm not sure if that overhead is worth worrying about given the already incurred logging overhead.) So, I think it's better to have a flag like PQTRACE_OUTPUT_TIMESTAMPS instead of PQTRACE_SUPPRESS_TIMESTAMPS, and the functions would look like:

void
PQtrace(PGconn *conn, FILE *stream)
{
PQtraceEx(conn, stream, 0);
}

void
PQtraceEx(PGconn *conn, FILE *stream, int flags)
{
... the new implementation in the patch
}

Remember to add PQtraceEx in exports.txt and make sure it builds on Windows with Visual Studio.

But... can you evaluate the overhead for getting timestamps and see if we can turn it on by default, or further, if we need such a flag and PQtraceEx()? For example, how about comparing the times to run the regression test with and without timestamps?

(30)
+/*
+ * Deallocate FE/BE message tracking memory.  We only do this because
+ * FE message can grow arbitrarily large, and skip it in the initial state,
+ * because it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}

What does the second if condition mean? If it's not necessary, change the comment accordingly.

(31)
@@ -1011,11 +1615,16 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		/* XXX comment */
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
 		return pqSendSome(conn, conn->outCount);
+	}

Remove the comment line.

(32)
+#define INT_MAX (2048/2) /* maximum array size */

INT_MAX is available via:
#include <limits.h>

(33)
 	/* realloc failed. Probably out of memory */
-	appendPQExpBufferStr(&conn->errorMessage,
-						 "cannot allocate memory for input buffer\n");
+	appendPQExpBuffer(&conn->errorMessage,
+					  "cannot allocate memory for input buffer\n");

This change appears to be irrelevant.

(34)
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+		conn->fe_msg->msg_type = type;
+}

Protocol version check is unnecessary because this tracing takes effect only in v3 protocol?

(35)
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					sizeof(pqFrontendMessage) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));

Align this size calculation with that in pqTraceInit().

(36)
+	/* append milliseconds */
+	sprintf(timestr + strlen(timestr), ".%03d", (int) (tval.tv_usec / 1000));

Output microsecond instead. As your example output shows, many lines show the exactly same timestamps and thus they are not distinguishable in time.

(37)
+static char *
+pqLogFormatTimestamp(char *timestr)
+{
...
+#define FORMATTED_TS_LEN 128
+	strftime(timestr, FORMATTED_TS_LEN,

Add an argument to this function that indicates the size of timestr. Define FORMATTED_TS_LEN globally in this source file, and have callers use it. That is, the code like:

+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),

becomes:

+		char		timestr[FORMATTED_TS_LEN];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr, sizeof(timestr)),
(38)
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);
+	}
+	else
+		fprintf(conn->Pfdebug, ">\t%s\t%d",
+				pqGetProtocolMsgType(conn->fe_msg->msg_type,
+									 MSGDIR_FROM_FRONTEND),
+				msgLen);

To reduce the code for easier modification, change the above code to something like:

+	char		timestr[128];
+	char		*timestr_p = "";
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr);
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			pqGetProtocolMsgType(conn->fe_msg->msg_type,
+								 MSGDIR_FROM_FRONTEND),
+			msgLen);

Regards
Takayuki Tsunakawa

#92tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: k.jamison@fujitsu.com (#89)
RE: libpq debug log

From: Jamison, Kirk/ジャミソン カーク <k.jamison@fujitsu.com>

I realized that existing libpq regression tests in src/test/examples do not
test PQtrace(). At the same time, no other functions Is calling PQtrace(),
so it's expected that by default if there are no compilation errors, then it
will pass all the tests. And it did.

So to check that the tracing feature is enabled and, as requested, outputs
a trace file, I temporarily modified a bit of testlibpq.c instead **based from
my current environment settings**, and ran the regression test.

To run the tracing code in much more extensive cases, can you try running the whole SQL regression test with the tracing enabled? That is, run "make check-world" in the top directory of the source tree.

To enable tracing in every connection, you can probably insert PQtrace() call just before the return statement here in connectDBComplete(). If you have enough disk space, it's better to accumulate the trace output by passing "a" to fopen().

switch (flag)
{
case PGRES_POLLING_OK:
return 1; /* success! */

Regards
Takayuki Tsunakawa

#93iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#91)
1 attachment(s)
RE: libpq debug log

Hi all,

Thank you Kirk san for creating the v14 patch.
I update the patch. I fixed all of Tsunakawa san's review comments.
I am trying to solve three bugs. Two bags were pointed out by Alvaro san
in a previous e-mail. And I found one bug.

From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Sent: Friday, January 22, 2021 6:53 AM

...

(5)
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char

*name, const char *value)

pgParameterStatus *pstatus;
pgParameterStatus *prev;

- if (conn->Pfdebug)
- fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' =

'%s'\n",

- name, value);
-

Where is this information output instead?

Hmm, seems to have been lost. I restored it, but didn't verify
the resulting behavior carefully.

I checked log output using Matsumura san's application app.c;
---
2021-01-27 11:29:49.718 < ParameterStatus 22 "application_name" ""
pqSaveParameterStatus: 'application_name' = ''
2021-01-27 11:29:49.718 < ParameterStatus 25 "client_encoding" "UTF8"
pqSaveParameterStatus: 'client_encoding' = 'UTF8'
---

New trace log prints "ParameterStatus" which report run-time parameter status.
And new log also prints parameter name and value in " StartupMessage " message.
We can know the parameter status using these protocol messages.
So I think "pqSaveParameterStatus:" is not necessary to output.

In StartupMessage, the protocol message is output as "UnknownMessage" like this.
[...] > UnknownMessage 38 '\x00\x03\x00\x00user\x00iwata\x00database\x00postgres\x00\x00'

To fix this, I want to move a protocol message without the first byte1 to a frontend message logging function.
I will work on this.

From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
Sent: Friday, January 22, 2021 10:38 PM

Ah, that was the reason for separation. Then, I'm fine with either
keeping the separation, or integrating the two functions in fe-misc.c
with PQuntrace() being also there. I kind of think the latter is
better, because of code readability and, because tracing facility is
not a connection-related reature so categorizing it as "misc" feels
natural.

Maybe we can create a new file specifically for this to avoid mixing
unrelated stuff in fe-misc.c -- say fe-trace.c where all this new
tracing stuff goes.

I moved PQtrace() from fe-connect.c to fe-misc.c.
And I agree with creating new file for new tracing functions. I will do this.

From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
Sent: Thursday, January 28, 2021 11:14 PM

...

On 2021-Jan-28, k.jamison@fujitsu.com wrote:

I realized that existing libpq regression tests in src/test/examples do not
test PQtrace(). At the same time, no other functions Is calling PQtrace(),
so it's expected that by default if there are no compilation errors, then it
will pass all the tests. And it did.

So to check that the tracing feature is enabled and, as requested, outputs
a trace file, I temporarily modified a bit of testlibpq.c instead **based from
my current environment settings**, and ran the regression test.

Yeah. It seems useful to build a real test harness based on the new
tracing functionality. And that is precisely why I added the option to
suppress timestamps: so that we can write trace files that do not vary
from run to run. That allows us to have an "expected" trace to which we
can compare the trace file produced by the actual run. I had the idea
that instead of a timestamp we would print a message counter. I didn't
implement that, however.

I think it is a useful suggestion. However I couldn't think of how to find out
if the logs were really in the correct order. And implementing this change looks very complicated.
So I would like to brush up this patch at first.

(29)
-void PQtrace(PGconn *conn, FILE *stream);
+void PQtrace(PGconn *conn, FILE *stream, int flags);

As I said before, I'm afraid we cannot change the signature of existing API
functions. Please leave the signature of this function unchanged, and add a
new function like PQtraceEx() that adds the flags argument.

I guess the purpose of adding the flag argument is to not output the timestamp
by default, because getting timestamps would incur significant overhead
(however, I'm not sure if that overhead is worth worrying about given the
already incurred logging overhead.) So, I think it's better to have a flag like
PQTRACE_OUTPUT_TIMESTAMPS instead of
PQTRACE_SUPPRESS_TIMESTAMPS, and the functions would look like:

void
PQtrace(PGconn *conn, FILE *stream)
{
PQtraceEx(conn, stream, 0);
}

void
PQtraceEx(PGconn *conn, FILE *stream, int flags)
{
... the new implementation in the patch
}

Remember to add PQtraceEx in exports.txt and make sure it builds on
Windows with Visual Studio.

I fixed just add new function.
I will look into similar changes so far and give PQtraceEx() a better name.

But... can you evaluate the overhead for getting timestamps and see if we can
turn it on by default, or further, if we need such a flag and PQtraceEx()? For
example, how about comparing the times to run the regression test with and
without timestamps?

I will check it. And decide which is better by default.
In my understanding, one of the purpose of this flag is to make sure
the log order is correct regardless of timestamp. (for example, it is useful auto-regression test)
I think PQtraceEx() and this flag is necessary as a developer option at least.

(30)
+/*
+ * Deallocate FE/BE message tracking memory.  We only do this because
+ * FE message can grow arbitrarily large, and skip it in the initial state,
+ * because it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}

What does the second if condition mean? If it's not necessary, change the
comment accordingly.

I fix the comment and remove second if.

(31)
@@ -1011,11 +1615,16 @@ pqSendSome(PGconn *conn, int len)
int
pqFlush(PGconn *conn)
{
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
if (conn->outCount > 0)
+	{
+		/* XXX comment */
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
return pqSendSome(conn, conn->outCount);
+	}

Remove the comment line.

I remove it.

(32)
+#define INT_MAX (2048/2) /* maximum array size */

INT_MAX is available via:
#include <limits.h>

I fixed it.

(33)
/* realloc failed. Probably out of memory */
-	appendPQExpBufferStr(&conn->errorMessage,
-						 "cannot allocate memory
for input buffer\n");
+	appendPQExpBuffer(&conn->errorMessage,
+					  "cannot allocate memory for input
buffer\n");

This change appears to be irrelevant.

I remove this code.

(34)
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+		conn->fe_msg->msg_type = type;
+}

Protocol version check is unnecessary because this tracing takes effect only in
v3 protocol?

Yes, in PQtrace(), it is checked whether protocol 2.0 or 3.0 and tracing works only in v3.0. I remove it.

(35)
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					sizeof(pqFrontendMessage) +
+					2 * conn->fe_msg->max_fields *
sizeof(pqFrontendMessageField));

Align this size calculation with that in pqTraceInit().

I fixed it like this;
+               conn->fe_msg =
+                       realloc(conn->fe_msg,
+                                       offsetof(pqFrontendMessage, fields) +
+                                       2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
(36)
+	/* append milliseconds */
+	sprintf(timestr + strlen(timestr), ".%03d", (int) (tval.tv_usec / 1000));

Output microsecond instead. As your example output shows, many lines
show the exactly same timestamps and thus they are not distinguishable in
time.

I changed %03d to %06d for microsecond.

(37)
+static char *
+pqLogFormatTimestamp(char *timestr)
+{
...
+#define FORMATTED_TS_LEN 128
+	strftime(timestr, FORMATTED_TS_LEN,

Add an argument to this function that indicates the size of timestr. Define
FORMATTED_TS_LEN globally in this source file, and have callers use it.
That is, the code like:

+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),

becomes:

+		char		timestr[FORMATTED_TS_LEN];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr,
sizeof(timestr)),

I fixed it.

(38)
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		char		timestr[128];
+
+		fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+				pqLogFormatTimestamp(timestr),
+
pqGetProtocolMsgType(conn->fe_msg->msg_type,
+
MSGDIR_FROM_FRONTEND),
+				msgLen);
+	}
+	else
+		fprintf(conn->Pfdebug, ">\t%s\t%d",
+
pqGetProtocolMsgType(conn->fe_msg->msg_type,
+
MSGDIR_FROM_FRONTEND),
+				msgLen);

To reduce the code for easier modification, change the above code to
something like:

+	char		timestr[128];
+	char		*timestr_p = "";
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr);
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			pqGetProtocolMsgType(conn->fe_msg->msg_type,
+
MSGDIR_FROM_FRONTEND),
+			msgLen);

I fixed it.

This patch should address the following:
1. fix 3 bugs
1.1 -1 output in "Query" message
1.2 two message output in "ReadyForQuery" message
1.3 "StartupMessage" output as " UnknownMessage "
2. creating new file for new tracing functions

Regards,
Aya Iwata
Fujitsu

Attachments:

v15-0001-libpq-trace.patchapplication/octet-stream; name=v15-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b7a8245..cad0c34 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,24 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Calls <function>PQtraceEx</function> to output with or without a timestamp
+      using <literal>flags</literal>.
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> &amp; <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>) is
+      true, then timestamp is not printed with each message.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..afbde7d 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceEx                 180
\ No newline at end of file
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583..d2796d2 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6760,27 +6760,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..14b8302 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -30,6 +30,7 @@
 
 #include "postgres_fe.h"
 
+#include <limits.h>
 #include <signal.h>
 #include <time.h>
 
@@ -51,13 +52,171 @@
 #include "libpq-int.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
+#include "pgtime.h"
 #include "port/pg_bswap.h"
 
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
 static int	pqPutMsgBytes(const void *buf, size_t len, PGconn *conn);
 static int	pqSendSome(PGconn *conn, int len);
 static int	pqSocketCheck(PGconn *conn, int forRead, int forWrite,
 						  time_t end_time);
 static int	pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time);
+static void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+static void pqStoreFeMsgStart(PGconn *conn, char type);
+static void pqLogFrontendMsg(PGconn *conn, int msgLen);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogMessageByte1(PGconn *conn, char v);
+static void pqLogMessageInt(PGconn *conn, int v, int length);
+static void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
 
 /*
  * PQlibVersion: return the libpq version number
@@ -68,7 +227,6 @@ PQlibVersion(void)
 	return PG_VERSION_NUM;
 }
 
-
 /*
  * pqGetc: get 1 character from the connection
  *
@@ -85,7 +243,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqLogMessageByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +259,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +297,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMessageString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -157,18 +314,19 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 	return pqGets_internal(buf, conn, false);
 }
 
-
 /*
  * pqPuts: write a null-terminated string to the current message
  */
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +347,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMessagenchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +367,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqLogMessagenchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +385,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +425,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqLogMessageInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +440,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +464,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +684,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqStoreFeMsgStart(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +720,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -600,6 +747,486 @@ pqPutMsgEnd(PGconn *conn)
 	return 0;
 }
 
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	PQtraceEx(conn, debug_port, 0);
+}
+
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (!debug_port)
+		return;
+	if (pqTraceInit(conn, flags))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+	/* Deallocate FE/BE message tracking memory. */
+	if (conn->fe_msg)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+bool
+pqTraceInit(PGconn *conn, int flags)
+{
+	conn->traceFlags = flags;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+static void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+static void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, Size ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec / 1000));
+
+	return timestr;
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+static void
+pqLogFrontendMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+
+	if ((conn->traceFlags & PQTRACE_OUTPUT_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			pqGetProtocolMsgType(conn->fe_msg->msg_type,
+								 MSGDIR_FROM_FRONTEND),
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqLogMessageString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMessagenchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ */
+static void
+pqLogMessageByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_OUTPUT_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			/*
+				* True type of message tagged '\0' is known when next 4 bytes
+				* is checked. So we delay printing message type to
+				* pqLogMessageInt()
+				*/
+			if (v != '\0')
+				fprintf(conn->Pfdebug, "%s\t",
+						pqGetProtocolMsgType((unsigned char) v,
+												MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+static void
+pqLogMessageInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			if (conn->be_msg->command == '\0')
+			{
+				char	   *message_type;
+
+				/*
+					* We delayed printing message type for special messages;
+					* they are complete now, so print them.
+					*/
+				if (conn->fe_msg->num_fields > 0)
+				{
+					int			message_addr;
+					uint32		result32;
+					int			result;
+
+					message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+					memcpy(&result32, conn->outBuffer + message_addr, 4);
+					result = (int) pg_ntoh32(result32);
+
+					if (result == NEGOTIATE_SSL_CODE)
+						message_type = "SSLRequest";
+					else if (result == NEGOTIATE_GSS_CODE)
+						message_type = "GSSRequest";
+					else
+						message_type = "StartupMessage";
+				}
+				else
+					message_type = "UnknownMessage";
+				fprintf(conn->Pfdebug, "%s ", message_type);
+			}
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+static void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+
 /* ----------
  * pqReadData: read more data, if any is available
  * Possible return values:
@@ -1011,11 +1638,15 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+		{
+			pqLogFrontendMsg(conn, -1);
+			fflush(conn->Pfdebug);
+		}
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..d1a74f1 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -156,7 +156,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +288,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +365,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +377,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +407,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +471,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1640,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index c266ad5..1bfbe19 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_OUTPUT_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceEx(PGconn *conn, FILE *debug_port, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db4983..eed4ac7 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -155,6 +155,14 @@ typedef struct
 	void	   *noticeProcArg;
 } PGNoticeHooks;
 
+/*
+ * Logging
+ */
+
+/* Forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 typedef struct PGEvent
 {
 	PGEventProc proc;			/* the function to call on events */
@@ -375,6 +383,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -672,6 +685,9 @@ extern int	pqPutInt(int value, size_t bytes, PGconn *conn);
 extern int	pqPutMsgStart(char msg_type, bool force_len, PGconn *conn);
 extern int	pqPutMsgEnd(PGconn *conn);
 extern int	pqReadData(PGconn *conn);
+extern bool pqTraceInit(PGconn *conn, int flags);
+extern void pqTraceUninit(PGconn *conn);
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
 extern int	pqFlush(PGconn *conn);
 extern int	pqWait(int forRead, int forWrite, PGconn *conn);
 extern int	pqWaitTimed(int forRead, int forWrite, PGconn *conn,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1d540fe..09244d6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#94'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#91)
Re: libpq debug log

On 2021-Jan-29, tsunakawa.takay@fujitsu.com wrote:

(30)
+/*
+ * Deallocate FE/BE message tracking memory.  We only do this because
+ * FE message can grow arbitrarily large, and skip it in the initial state,
+ * because it's likely pointless.
+ */
+void
+pqTraceUninit(PGconn *conn)
+{
+	if (conn->fe_msg &&
+		conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}

What does the second if condition mean? If it's not necessary, change the comment accordingly.

The rationale for that second condition is this: if the memory allocated
is the initial size, we don't free memory, because it would just be
allocated of the same size next time, and that size is not very big, so
it's not a big deal if we just let it be, so that it is reused if we
call PQtrace() again later. However, if the allocated size is larger
than default, then it is possible that some previous tracing run has
enlarged the trace struct to a very large amount of memory, and we don't
want to leave that in place.

--
�lvaro Herrera Valdivia, Chile

#95tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#93)
RE: libpq debug log
(39)
+      of tracing.  If (<literal>flags</literal> &amp; <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>) is
+      true, then timestamp is not printed with each message.

The flag name (OUTPUT) and its description (not printed) doesn't match.

I think you can use less programmatical expression like "If <literal>flags</literal> contains <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>".

(40)
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
+PQtraceEx 180
\ No newline at end of file

What's the second line? Isn't the file missing an empty line at the end?

(41)
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{
+	if (conn == NULL)
+		return;
...
+	if (!debug_port)
+		return;

The if should better be as follows to match the style of existing surrounding code.

+ if (debug_port == NULL)

(42)
+pqLogFormatTimestamp(char *timestr, Size ts_len)

I think you should use int or size_t instead of Size here, because other libpq code uses them. int should be enough. If the compiler gives warnings, prepend "(int)" before sizeof() at call sites.

(43)
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec / 1000));

"/ 1000" should be removed.

(44)
+	if ((conn->traceFlags & PQTRACE_OUTPUT_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));

== 0 should be removed.

Regards
Takayuki Tsunakawa

#96tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: 'Alvaro Herrera' (#94)
RE: libpq debug log

From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>

+ conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)

The rationale for that second condition is this: if the memory allocated
is the initial size, we don't free memory, because it would just be
allocated of the same size next time, and that size is not very big, so
it's not a big deal if we just let it be, so that it is reused if we
call PQtrace() again later. However, if the allocated size is larger
than default, then it is possible that some previous tracing run has
enlarged the trace struct to a very large amount of memory, and we don't
want to leave that in place.

Ah, understood. In that case, num_fields should be max_fields.

This has reminded me that freePGconn() should free be_msg and fe_msg.

Regards
Takayuki Tsunakawa

#97Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#95)
Re: libpq debug log

At Wed, 3 Feb 2021 01:26:41 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

(41)
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{
+	if (conn == NULL)
+		return;
...
+	if (!debug_port)
+		return;

The if should better be as follows to match the style of existing surrounding code.

+ if (debug_port == NULL)

I don't have particular preference here, but FWIW the current
PQuntrace is doing this:

void
PQuntrace(PGconn *conn)
{
if (conn == NULL)
return;
if (conn->Pfdebug)
{
fflush(conn->Pfdebug);

(44)
+	if ((conn->traceFlags & PQTRACE_OUTPUT_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));

== 0 should be removed.

Looking the doc mentioned in the comment #39:

+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> &amp; <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>) is
+      true, then timestamp is not printed with each message.

PQTRACE_OUTPUT_TIMESTAMPS is designed to *inhibit* timestamps from
being prepended. If that is actually intended, the symbol name should
be PQTRACE_NOOUTPUT_TIMESTAMP. Otherwise, the doc need to be fixed.

By the way removing "== 0" makes it difficult to tell whether the
condition is correct or not; I recommend to use '!= 0" rather than
removing '== 0'.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#98Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Kyotaro Horiguchi (#97)
Re: libpq debug log

On 2021-Feb-03, Kyotaro Horiguchi wrote:

Looking the doc mentioned in the comment #39:

+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> &amp; <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>) is
+      true, then timestamp is not printed with each message.

PQTRACE_OUTPUT_TIMESTAMPS is designed to *inhibit* timestamps from
being prepended. If that is actually intended, the symbol name should
be PQTRACE_NOOUTPUT_TIMESTAMP. Otherwise, the doc need to be fixed.

I'm pretty sure I named the flag PQTRACE_SUPPRESS_TIMESTAMP (and I
prefer SUPPRESS to NOOUTPUT), because the idea is that the timestamp is
printed by default. I think that's the sensible decision: applications
prefer to have timestamps, even if there's a tiny bit of overhead. We
don't want to force them to pass a flag for that. We only want the
no-timestamp behavior in order to be able to use it for libpq internal
testing.

--
�lvaro Herrera 39�49'30"S 73�17'W
"Someone said that it is at least an order of magnitude more work to do
production software than a prototype. I think he is wrong by at least
an order of magnitude." (Brian Kernighan)

#99'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#95)
Re: libpq debug log

On 2021-Feb-03, tsunakawa.takay@fujitsu.com wrote:

(39)
+      of tracing.  If (<literal>flags</literal> &amp; <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>) is
+      true, then timestamp is not printed with each message.

The flag name (OUTPUT) and its description (not printed) doesn't match.

I think you can use less programmatical expression like "If <literal>flags</literal> contains <literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>".

I copied the original style from elsewhere in the manual.

(41)
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{

I'm not really sure about making this a separate API call. We could just
make it PQtrace() and increment the libpq so version. I don't think
it's a big deal, frankly.

--
�lvaro Herrera 39�49'30"S 73�17'W
"El conflicto es el camino real hacia la uni�n"

#100'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#96)
Re: libpq debug log

On 2021-Feb-03, tsunakawa.takay@fujitsu.com wrote:

From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>

+ conn->fe_msg->num_fields != DEF_FE_MSGFIELDS)

The rationale for that second condition is this: if the memory allocated
is the initial size, we don't free memory, because it would just be
allocated of the same size next time, and that size is not very big, so
it's not a big deal if we just let it be, so that it is reused if we
call PQtrace() again later. However, if the allocated size is larger
than default, then it is possible that some previous tracing run has
enlarged the trace struct to a very large amount of memory, and we don't
want to leave that in place.

Ah, understood. In that case, num_fields should be max_fields.

Oh, of course.

--
�lvaro Herrera 39�49'30"S 73�17'W
"Just treat us the way you want to be treated + some extra allowance
for ignorance." (Michael Brusser)

#101tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Alvaro Herrera (#98)
RE: libpq debug log

From: Alvaro Herrera <alvherre@alvh.no-ip.org>

I'm pretty sure I named the flag PQTRACE_SUPPRESS_TIMESTAMP (and I
prefer SUPPRESS to NOOUTPUT), because the idea is that the timestamp is
printed by default. I think that's the sensible decision: applications
prefer to have timestamps, even if there's a tiny bit of overhead. We
don't want to force them to pass a flag for that. We only want the
no-timestamp behavior in order to be able to use it for libpq internal
testing.

Agreed. It makes sense to print timestamps by default because this feature will be used to diagnose slowness outside the database server. (I misunderstood the motivation to introduce the flag).

Regards
Takayuki Tsunakawa

#102tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: 'Alvaro Herrera' (#99)
RE: libpq debug log

From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>

(41)
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{

I'm not really sure about making this a separate API call. We could just
make it PQtrace() and increment the libpq so version. I don't think
it's a big deal, frankly.

If we change the function signature, we have to bump the so major version and thus soname. The libpq's current so major version is 5, which hasn't been changed since 2006. I'm hesitant to change it for this feature. If you think we can bump the version to 6, I think we can go.

https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
--------------------------------------------------
try to make sure that your libraries are either backwards-compatible or that you've incremented the version number in the soname every time you make an incompatible change.

When a new version of a library is binary-incompatible with the old one the soname needs to change. In C, there are four basic reasons that a library would cease to be binary compatible:

1. The behavior of a function changes so that it no longer meets its original specification,

2. Exported data items change (exception: adding optional items to the ends of structures is okay, as long as those structures are only allocated within the library).

3. An exported function is removed.

4. The interface of an exported function changes.
--------------------------------------------------

Regards
Takayuki Tsunakawa

#103iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#95)
1 attachment(s)
RE: libpq debug log

HI all,

I update the patch.
I modified code according to review comments of Tsunakawa san and Horiguchi san.

And I fixed some bugs.

This patch should address the following:
1. fix 3 bugs
1.1 -1 output in "Query" message

The cause of this bug is that it call in pqFlush() function before flushing.
The purpose of calling pqLogFrontendMsg() here is to log the data that was in the buffer but not logged before flushing.
The second argument of pqLogFrontendMsg() is message length, but we can't find it in pqFlush().
Therefor, -1 was set here as a second argument and it was logged.
I thought about keeping the length as fields, but from the output Kirk san tried, this doesn't seem to happen.
Also, the message from the frontend to the backend calls pqPutMsgEnd () in advance, so the message should already be logged.
So I deleted this call.

1.2 two message output in "ReadyForQuery" message

In the pqParseInput3 (), when the state becomes PGASYNC_IDLE, it returns to the caller.
After that, pqParseInput3 () is called again and protocol message and length are output again by calling pqGetc(), pqGetInt().
To limit this, set the skipLogging flag when the status become PGASYNC_IDLE and fixed to skip the next pqGetc(), pqGetInt().

1.3 "StartupMessage" output as " UnknownMessage "

I moved the code that handles the output of "SSLRequest", "StartupMessage", etc.
to pqLogFrontendMsg() which is the function that outputs the front-end message.

2. creating new file for new tracing functions

I create new files libpq-logging.c and libpq-logging.h

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Wednesday, February 3, 2021 10:27 AM

(39)
+      of tracing.  If (<literal>flags</literal> &amp;
<literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>) is
+      true, then timestamp is not printed with each message.

The flag name (OUTPUT) and its description (not printed) doesn't match.

I changed flag name to PQTRACE_SUPPRESS_TIMESTAMPS.

I think you can use less programmatical expression like "If
<literal>flags</literal> contains
<literal>PQTRACE_OUTPUT_TIMESTAMPS</literal>".

I fixed.

(40)
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
+PQtraceEx 180
\ No newline at end of file

What's the second line? Isn't the file missing an empty line at the end?

I delete it.

(41)
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags) {
+	if (conn == NULL)
+		return;
...
+	if (!debug_port)
+		return;

The if should better be as follows to match the style of existing surrounding
code.

+ if (debug_port == NULL)

I fixed it.

(42)
+pqLogFormatTimestamp(char *timestr, Size ts_len)

I think you should use int or size_t instead of Size here, because other libpq
code uses them. int should be enough. If the compiler gives warnings,
prepend "(int)" before sizeof() at call sites.

I fixed it.

(43)
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec /
+1000));

"/ 1000" should be removed.

I removed it.

(44)
+	if ((conn->traceFlags & PQTRACE_OUTPUT_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr,
sizeof(timestr));

== 0 should be removed.

I left it, referring Horiguchi san's comment.

Regards,
Aya Iwata

Attachments:

v16-0001-libpq-trace.patchapplication/octet-stream; name=v16-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b7a8245..b59e393 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,24 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Calls <function>PQtraceEx</function> to output with or without a timestamp
+      using <literal>flags</literal>.
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),
+      then timestamp is not printed with each message.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677e..b703e0c 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-logging.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-logging.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir)/libpq-logging.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..afbde7d 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceEx                 180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583..e41cd63 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4015,6 +4015,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6760,27 +6764,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..81ed88f 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-logging.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -84,8 +85,8 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	if (conn->Pfdebug && !conn->skipLogging)
+			pqLogMessageByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqStoreFrontendMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqLogMessageString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqStoreFrontendMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqLogMessagenchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqLogMessagenchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqStoreFrontendMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -278,8 +268,8 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	if (conn->Pfdebug && !conn->skipLogging)
+		pqLogMessageInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqStoreFrontendMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +528,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqStoreFeMsgStart(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +564,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqLogFrontendMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -1011,11 +1002,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..f5ef58e 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -81,6 +81,9 @@ pqParseInput3(PGconn *conn)
 		if (pqGetInt(&msgLength, 4, conn))
 			return;
 
+		if (conn->Pfdebug && conn->skipLogging)
+			conn->skipLogging = 0;
+
 		/*
 		 * Try to validate message type/length here.  A length less than 4 is
 		 * definitely broken.  Large lengths should only be believed for a few
@@ -156,7 +159,12 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				/* Set flag to notice we already log first byte1 and length */
+				if (conn->Pfdebug)
+					conn->skipLogging = 1;
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +291,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +368,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +380,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +410,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyBreakLine(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +474,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyBreakLine(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1643,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyBreakLine(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index c266ad5..a1300ef 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceEx(PGconn *conn, FILE *debug_port, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db4983..416b577 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -22,6 +22,7 @@
 
 /* We assume libpq-fe.h has already been included. */
 #include "libpq-events.h"
+#include "libpq-logging.h"
 
 #include <time.h>
 #ifndef WIN32
@@ -375,6 +376,12 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+	int			skipLogging;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-logging.c b/src/interfaces/libpq/libpq-logging.c
new file mode 100644
index 0000000..abb9576
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.c
@@ -0,0 +1,619 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-logging.c
+ *	  functions for supporting the libpq "logging"
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-logging.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-logging.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static bool pqTraceInit(PGconn *conn, int flags);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+							PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqLogFrontendMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	PQtraceEx(conn, debug_port, 0);
+}
+
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn, flags))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+	/* Deallocate FE/BE message tracking memory. */
+	if (conn->fe_msg &&
+		/*
+		 * If fields is allocated the initial size, we reuse it next time,
+		 * because it would be allocated same size and the size is not big.
+		 */
+			conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn, int flags)
+{
+	conn->traceFlags = flags;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->skipLogging = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+
+	return timestr;
+}
+
+/*
+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqLogFrontendMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+	const char	   *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+			* We delayed printing message type for special messages;
+			* they are complete now, so print them.
+			*/
+		if (conn->fe_msg->num_fields > 0)
+		{
+			int			message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+			memcpy(&result32, conn->outBuffer + message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+			else
+				message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqLogMessageString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqLogMessagenchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ */
+void
+pqLogMessageByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			/*
+				* True type of message tagged '\0' is known when next 4 bytes
+				* is checked. So we delay printing message type to
+				* pqLogMessageInt()
+				*/
+			if (v != '\0')
+				fprintf(conn->Pfdebug, "%s\t",
+						pqGetProtocolMsgType((unsigned char) v,
+												MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqLogMessageInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+
+/*
+ * pqLogMessageString: output a null-terminated string to the log
+ */
+void
+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-logging.h b/src/interfaces/libpq/libpq-logging.h
new file mode 100644
index 0000000..e274267
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-logging.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq logging.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-logging.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_LOGGING_H
+#define LIBPQ_LOGGING_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
+
+extern void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqStoreFeMsgStart(PGconn *conn, char type);
+extern void pqLogFrontendMsg(PGconn *conn, int msgLen);
+extern void pqLogMessageByte1(PGconn *conn, char v);
+extern void pqLogMessageInt(PGconn *conn, int v, int length);
+extern void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_LOGGING_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1d540fe..09244d6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#104tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#103)
RE: libpq debug log

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

I update the patch.
I modified code according to review comments of Tsunakawa san and
Horiguchi san.

I confirmed that all the previous feedback was reflected. Here are some minor comments:

(45)
void PQtrace(PGconn *conn, FILE *stream);
</synopsis>
</para>

+     <para>
+      Calls <function>PQtraceEx</function> to output with or without a timestamp
+      using <literal>flags</literal>.
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),
+      then timestamp is not printed with each message.

The description of PQtrace() should be written independent of PQtraceEx(). It is an unnecessary implementation detail to the user that PQtrace() calls PQtraceEx() internally. Plus, a separate entry for PQtraceEx() needs to be added.

(46)

If skipLogging is intended for use with backend -> frontend messages only, shouldn't it be placed in conn->b_msg?

(47)
+	/* Deallocate FE/BE message tracking memory. */
+	if (conn->fe_msg &&
+		/*
+		 * If fields is allocated the initial size, we reuse it next time,
+		 * because it would be allocated same size and the size is not big.
+		 */
+			conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)

I'm not completely sure if other places interpose a block comment like this between if/for/while conditions, but I think it's better to put the comment before if.

Regards
Takayuki Tsunakawa

#105Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#104)
Re: libpq debug log

At Tue, 9 Feb 2021 02:26:25 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

I update the patch.
I modified code according to review comments of Tsunakawa san and
Horiguchi san.

I confirmed that all the previous feedback was reflected. Here are some minor comments:

(45)
void PQtrace(PGconn *conn, FILE *stream);
</synopsis>
</para>

+     <para>
+      Calls <function>PQtraceEx</function> to output with or without a timestamp
+      using <literal>flags</literal>.
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),
+      then timestamp is not printed with each message.

The description of PQtrace() should be written independent of PQtraceEx(). It is an unnecessary implementation detail to the user that PQtrace() calls PQtraceEx() internally. Plus, a separate entry for PQtraceEx() needs to be added.

This looks like a fusion of PQtrace and PQtraceEX. By the way, the
timestamp flag is needed at log emittion. So we can change the state
anytime.

PQtrace(conn, of);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);
<logging without timestamps>
PQtraceSetFlags(conn, 0);
<logging with timestamps>
..

(46)

If skipLogging is intended for use with backend -> frontend messages only, shouldn't it be placed in conn->b_msg?

The name skipLogging is somewhat obscure. The flag doesn't inhibit all
logs from being emitted. It seems like it represents how far bytes
the logging mechanism consumed for the limited cases. Thus, I think it
can be a cursor variable like inCursor.

If we have conn->be_msg->inLogged, for example, pqGetc and
pqLogMessageByte1() are written as the follows.

pqGetc(char *result, PGconn *conn)
{
if (conn->inCursor >= conn->inEnd)
return EOF;

*result = conn->inBuffer[conn->inCursor++];

if (conn->Pfdebug)
pqLogMessageByte1(conn, *result);

return 0;
}

pqLogMessageByte1(...)
{
switch()
{
case LOG_FIRST_BYTE:
/* No point in logging already logged bytes */
if (conn->be_msg->inLogged >= conn->inCursor)
return;
...
}
conn->be_msg->inLogged = conn->inCursor;
}

(pqCheckInBufferSpace needs to adjust inLogged.)

I'm not sure this is easier to read than the skipLogging.

(47)
+	/* Deallocate FE/BE message tracking memory. */
+	if (conn->fe_msg &&
+		/*
+		 * If fields is allocated the initial size, we reuse it next time,
+		 * because it would be allocated same size and the size is not big.
+		 */
+			conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)

I'm not completely sure if other places interpose a block comment like this between if/for/while conditions, but I think it's better to put the comment before if.

(s/is/are/)

Agreed. At least it doesn't look good.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#106tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#105)
RE: libpq debug log

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

(45)

This looks like a fusion of PQtrace and PQtraceEX. By the way, the
timestamp flag is needed at log emittion. So we can change the state
anytime.

PQtrace(conn, of);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);
<logging without timestamps>
PQtraceSetFlags(conn, 0);
<logging with timestamps>

I find this better because the application does not have to call PQuntrace() and PQtrace() again to enable/disable timestamp output, which requires passing the FILE pointer again. (I don't imagine applications would repeatedly turn logging on and off in practice, though.)

(46)

If skipLogging is intended for use with backend -> frontend messages only,

shouldn't it be placed in conn->b_msg?

The name skipLogging is somewhat obscure. The flag doesn't inhibit all
logs from being emitted. It seems like it represents how far bytes
the logging mechanism consumed for the limited cases. Thus, I think it
can be a cursor variable like inCursor.

If we have conn->be_msg->inLogged, for example, pqGetc and
pqLogMessageByte1() are written as the follows.

pqGetc(char *result, PGconn *conn)
{
if (conn->inCursor >= conn->inEnd)
return EOF;

*result = conn->inBuffer[conn->inCursor++];

if (conn->Pfdebug)
pqLogMessageByte1(conn, *result);

return 0;
}

pqLogMessageByte1(...)
{
switch()
{
case LOG_FIRST_BYTE:
/* No point in logging already logged bytes */
if (conn->be_msg->inLogged >= conn->inCursor)
return;
...
}
conn->be_msg->inLogged = conn->inCursor;
}

This looks better design because stuff like skipLogging is an internal state of logging facility whose use should be restricted to fe-logging.c.

(pqCheckInBufferSpace needs to adjust inLogged.)

I'm not sure this is easier to read than the skipLogging.

I'd like Iwata-san to evaluate this and decide whether to take this approach or the current one.

Regards
Takayuki Tsunakawa

#107Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: iwata.aya@fujitsu.com (#103)
Re: libpq debug log

At Mon, 8 Feb 2021 14:57:47 +0000, "iwata.aya@fujitsu.com" <iwata.aya@fujitsu.com> wrote in

I update the patch.
I modified code according to review comments of Tsunakawa san and Horiguchi san.

And I fixed some bugs.

Thanks for the new version.

+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;

This is halfly exposed to other part of libpq. Specifically only
MSGDIR_FROM_BACKEND is used in fe-misc.c and only for
pgLogMessageString and pqLogMessagenchar. I would suggest to hide this
enum from fe-misc.c.

Looking into pqLogMessageString,

+pqLogMessageString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}

The only part that shared by both be and fe is the fprintf(). I think
it can be naturally split into separate functions for backend and
frontend messages.

Looking into pqLogMessagenchar,

+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the log
+ */
+void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */

# What is this???

+	}
    .. shared part
+}

pqLogMessagenchar is the sole caller of pqLogBinaryMsg. So we can
refactor the two functions and have pqLogMessagenchar_for_be and
_for_fe without a fear of the side effect to other callers. (The
names are discussed below.)

+typedef enum PGLogState
+{

This is libpq-logging.c internal type. It is not needed to be exposed.

+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
+extern void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)+extern void pqStoreFeMsgStart(PGconn *conn, char type);
+extern void pqLogFrontendMsg(PGconn *conn, int msgLen);
+extern void pqLogMessageByte1(PGconn *conn, char v);
+extern void pqLogMessageInt(PGconn *conn, int v, int length);
+extern void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);

The API functions looks like randomly/inconsistently named and
designed. I think that API should be in more structurally designed.

The comments about individual function names follow.

+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}

Differently from the comment, this function doesn't work in a
conditional way nor in a forceful way. It is just putting a new line
and resetting the backend message variables. I would name this as
pqTrace(Finish|Close)BeMsgLog().

+/*
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{

I would name this as pqTrace(Store/Append)FeMsg().

+void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}

The name says that "stores the message "start"". But it actually
stores the message type. I would name this as
pqTraceSetFeMsgType(). Or in contrast to pqTraceFinishBeMsgLog, this
can be named as pqTrace(Start|Begin|Init)BeMsgLog().

+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqLogFrontendMsg(PGconn *conn, int msgLen)

I would name this as pqTraceEmitFeMsgLog().

+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the log

I would name this as pqTraceEmitBeByteLog(), pqTraceEmitBeIntLog()
respectively.

+ * pqLogMessageString: output a null-terminated string to the log

This function is used for both directions. pqTraceEmitStringLog(). If it is split into fe and be parts, they would be pqTraceEmit(Be|Fe)StringLog().

+ * pqLogMessagenchar: output a string of exactly len bytes message to the log

This logs a byte sequence in hexadecimals. I would name that as
pqTraceEmitBytesLog(). Or pqTraceEmit(Be|Fe)BytesLog().

...I finish once here.

Is there any thoughts? Optinions on the namings?

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#108Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#106)
Re: libpq debug log

At Tue, 9 Feb 2021 08:10:19 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

(45)

This looks like a fusion of PQtrace and PQtraceEX. By the way, the
timestamp flag is needed at log emittion. So we can change the state
anytime.

PQtrace(conn, of);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);
<logging without timestamps>
PQtraceSetFlags(conn, 0);
<logging with timestamps>

I find this better because the application does not have to call PQuntrace() and PQtrace() again to enable/disable timestamp output, which requires passing the FILE pointer again. (I don't imagine applications would repeatedly turn logging on and off in practice, though.)

Thanks and Yes, I also don't imagine that users change the timestmp
state repeatedly:p It's just a example.

(46)

If skipLogging is intended for use with backend -> frontend messages only,

shouldn't it be placed in conn->b_msg?

The name skipLogging is somewhat obscure. The flag doesn't inhibit all
logs from being emitted. It seems like it represents how far bytes
the logging mechanism consumed for the limited cases. Thus, I think it
can be a cursor variable like inCursor.

If we have conn->be_msg->inLogged, for example, pqGetc and
pqLogMessageByte1() are written as the follows.

pqGetc(char *result, PGconn *conn)
{
if (conn->inCursor >= conn->inEnd)
return EOF;

*result = conn->inBuffer[conn->inCursor++];

if (conn->Pfdebug)
pqLogMessageByte1(conn, *result);

return 0;
}

pqLogMessageByte1(...)
{
switch()
{
case LOG_FIRST_BYTE:
/* No point in logging already logged bytes */
if (conn->be_msg->inLogged >= conn->inCursor)
return;
...
}
conn->be_msg->inLogged = conn->inCursor;
}

This looks better design because stuff like skipLogging is an internal state of logging facility whose use should be restricted to fe-logging.c.

(pqCheckInBufferSpace needs to adjust inLogged.)

I'm not sure this is easier to read than the skipLogging.

I'd like Iwata-san to evaluate this and decide whether to take this approach or the current one.

+1

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#109'Alvaro Herrera'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#102)
Re: libpq debug log

On 2021-Feb-04, tsunakawa.takay@fujitsu.com wrote:

From: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>

(41)
+void
+PQtraceEx(PGconn *conn, FILE *debug_port, int flags)
+{

I'm not really sure about making this a separate API call. We could just
make it PQtrace() and increment the libpq so version. I don't think
it's a big deal, frankly.

If we change the function signature, we have to bump the so major version and thus soname. The libpq's current so major version is 5, which hasn't been changed since 2006. I'm hesitant to change it for this feature. If you think we can bump the version to 6, I think we can go.

I think it's pretty clear we would not change the so-version for this
feature.

--
�lvaro Herrera 39�49'30"S 73�17'W

#110Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Kyotaro Horiguchi (#105)
Re: libpq debug log

On 2021-Feb-09, Kyotaro Horiguchi wrote:

This looks like a fusion of PQtrace and PQtraceEX. By the way, the
timestamp flag is needed at log emittion. So we can change the state
anytime.

PQtrace(conn, of);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);

I like this idea better than PQtraceEx().

--
�lvaro Herrera 39�49'30"S 73�17'W

#111iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Kyotaro Horiguchi (#107)
1 attachment(s)
RE: libpq debug log

Hi all,

Thank you for your review. I update patch to v17.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Tuesday, February 9, 2021 11:26 AM
(45)

...

The description of PQtrace() should be written independent of PQtraceEx().
It is an unnecessary implementation detail to the user that PQtrace() calls
PQtraceEx() internally. Plus, a separate entry for PQtraceEx() needs to be
added.

I add PQtraceSetFlags() description instead of PQtraceEx() in response to Horiguchi san's suggestion.

(46)

If skipLogging is intended for use with backend -> frontend messages only,
shouldn't it be placed in conn->b_msg?

I moved skip flag to be_msg.

(47)

...

I'm not completely sure if other places interpose a block comment like this
between if/for/while conditions, but I think it's better to put the comment
before if.

I moved this comment to before if.

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Tuesday, February 9, 2021 5:26 PM

+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;

This is halfly exposed to other part of libpq. Specifically only
MSGDIR_FROM_BACKEND is used in fe-misc.c and only for
pgLogMessageString and pqLogMessagenchar. I would suggest to hide this
enum from fe-misc.c.

Looking into pqLogMessageString,

+pqLogMessageString(PGconn *conn, const char *v, int length,
PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND &&
conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}

The only part that shared by both be and fe is the fprintf(). I think
it can be naturally split into separate functions for backend and
frontend messages.

Looking into pqLogMessagenchar,

+/*
+ * pqLogMessagenchar: output a string of exactly len bytes message to the
log
+ */
+void
+pqLogMessagenchar(PGconn *conn, const char *v, int len, PGCommSource
commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource
source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND &&
conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */

# What is this???

+	}
.. shared part
+}

pqLogMessagenchar is the sole caller of pqLogBinaryMsg. So we can
refactor the two functions and have pqLogMessagenchar_for_be and
_for_fe without a fear of the side effect to other callers. (The
names are discussed below.)

Thank you for your advice on refactoring.
Separating pqLogMessagenchar() into pqLogMessagenchar_for_be and pqLogMessagenchar_for_fe
seemed like adding more similar code. So I didn't work on it.

+typedef enum PGLogState
+{

This is libpq-logging.c internal type. It is not needed to be exposed.

I fixed it.

+extern void pqTraceForcelyBreakLine(int size, PGconn *conn);
+extern void pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type,
int length)+extern void pqStoreFeMsgStart(PGconn *conn, char type);
+extern void pqLogFrontendMsg(PGconn *conn, int msgLen);
+extern void pqLogMessageByte1(PGconn *conn, char v);
+extern void pqLogMessageInt(PGconn *conn, int v, int length);
+extern void pqLogMessageString(PGconn *conn, const char *v, int length,
+							   PGCommSource
commsource);
+extern void pqLogMessagenchar(PGconn *conn, const char *v, int length,
+							  PGCommSource
commsource);

The API functions looks like randomly/inconsistently named and
designed. I think that API should be in more structurally designed.

The comments about individual function names follow.

Thank you for your advice. I changed these functions name.

+/*
+ * pqTraceForcelyBreakLine:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyBreakLine(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}

Differently from the comment, this function doesn't work in a
conditional way nor in a forceful way. It is just putting a new line
and resetting the backend message variables. I would name this as
pqTrace(Finish|Close)BeMsgLog().

This function can be used when a connection is lost or when the copyData result is ignored according to the original code.
It is called to reset halfway logging. So I want to know that it will be forced to quit.
Therefore, I changed the name as pqTraceForcelyFinishBeMsgLog().

+/* 
+ * pqStoreFrontendMsg
+ *		Keep track of a from-frontend message that was just written
to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message
length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqStoreFrontendMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{

I would name this as pqTrace(Store/Append)FeMsg().

I renamed it to pqTraceStoreFeMsg(). Because people who do not know this feature is confused due to the name.
It is difficult to understand why store message during the logging.

+void
+pqStoreFeMsgStart(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}

The name says that "stores the message "start"". But it actually
stores the message type. I would name this as
pqTraceSetFeMsgType(). Or in contrast to pqTraceFinishBeMsgLog, this
can be named as pqTrace(Start|Begin|Init)BeMsgLog().

I think pqTraceSetFeMsgType() is better because this function's behavior is just set first byte1 message from Frontend.

+ * pqLogFrontendMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqLogFrontendMsg(PGconn *conn, int msgLen)

I would name this as pqTraceEmitFeMsgLog().

I would name this as pqTraceLogFeMsg().
It is shorter and we can easy to understand what this function do.

+ * pqLogMessageByte1: output 1 char from-backend message to the log
+ * pqLogMessageInt: output a 2- or 4-byte integer from-backend msg to the
log

I would name this as pqTraceEmitBeByteLog(), pqTraceEmitBeIntLog()
respectively.

I think log and emit would mean the same thing in this case. And log is more easy to understand in this case.
So I changed name to pqTraceLogBeMsgByte1() and pqTraceLogBeMsgInt().

+ * pqLogMessageString: output a null-terminated string to the log

This function is used for both directions. pqTraceEmitStringLog(). If it is split
into fe and be parts, they would be pqTraceEmit(Be|Fe)StringLog().

I changed name to pqTraceLogMsgString().

+ * pqLogMessagenchar: output a string of exactly len bytes message to the
log

This logs a byte sequence in hexadecimals. I would name that as
pqTraceEmitBytesLog(). Or pqTraceEmit(Be|Fe)BytesLog().

It is named to refer to pqGetnchar/pqPutnchar. So I changed this name to pqTraceLogMsgnchar().

...I finish once here.

Thank you for your review.

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Tuesday, February 9, 2021 5:30 PM

PQtrace(conn, of);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS); <logging

without

timestamps> PQtraceSetFlags(conn, 0); <logging with timestamps>

I find this better because the application does not have to call
PQuntrace() and PQtrace() again to enable/disable timestamp output,
which requires passing the FILE pointer again. (I don't imagine
applications would repeatedly turn logging on and off in practice,
though.)

Thanks and Yes, I also don't imagine that users change the timestmp state
repeatedly:p It's just a example.

I implemented PQtraceSetFlags() function. And add documentation of PQtraceSetFlags().

The name skipLogging is somewhat obscure. The flag doesn't inhibit
all logs from being emitted. It seems like it represents how far
bytes the logging mechanism consumed for the limited cases. Thus, I
think it can be a cursor variable like inCursor.

...

I'm not sure this is easier to read than the skipLogging.

I'd like Iwata-san to evaluate this and decide whether to take this approach

or the current one.

+1

I thought this idea was very good and tried it easily. But this didn't work...
So I didn't work on this implementation because the code can be more complex.

Regards
Aya Iwata
Fujits

Attachments:

v17-0001-libpq-trace.patchapplication/octet-stream; name=v17-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b7a8245..efc2fba 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5881,12 +5881,17 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Calls <function>PQtraceSetFlags</function> to output with or without a timestamp.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -5901,6 +5906,27 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Determine to output tracing with or without a timestamp to a debugging file stream.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),
+      then timestamp is not printed with each message. The default is output with timestamp.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677e..b703e0c 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-logging.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-logging.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir)/libpq-logging.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8ca0583..e41cd63 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4015,6 +4015,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6760,27 +6764,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..59cc1fe 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-logging.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -84,8 +85,8 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+	if (conn->Pfdebug && !conn->be_msg->skipLogging)
+			pqTraceLogBeMsgByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceLogMsgString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceLogMsgnchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -278,8 +268,8 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+	if (conn->Pfdebug && !conn->be_msg->skipLogging)
+		pqTraceLogBeMsgInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +528,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +564,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceLogFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -1011,11 +1002,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..d32daf7 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -81,6 +81,9 @@ pqParseInput3(PGconn *conn)
 		if (pqGetInt(&msgLength, 4, conn))
 			return;
 
+		if (conn->Pfdebug && conn->be_msg->skipLogging)
+			conn->be_msg->skipLogging = 0;
+
 		/*
 		 * Try to validate message type/length here.  A length less than 4 is
 		 * definitely broken.  Large lengths should only be believed for a few
@@ -156,7 +159,11 @@ pqParseInput3(PGconn *conn)
 		{
 			/* If not IDLE state, just wait ... */
 			if (conn->asyncStatus != PGASYNC_IDLE)
+			{
+				if (conn->Pfdebug)
+					conn->be_msg->skipLogging = 1;
 				return;
+			}
 
 			/*
 			 * Unexpected message in IDLE state; need to recover somehow.
@@ -283,6 +290,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +367,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +379,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +409,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +473,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1642,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index effe0cc..cdbaa2a 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 4db4983..99ac8dd 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -22,6 +22,7 @@
 
 /* We assume libpq-fe.h has already been included. */
 #include "libpq-events.h"
+#include "libpq-logging.h"
 
 #include <time.h>
 #ifndef WIN32
@@ -375,6 +376,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-logging.c b/src/interfaces/libpq/libpq-logging.c
new file mode 100644
index 0000000..ad981f8
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.c
@@ -0,0 +1,623 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-logging.c
+ *	  functions for supporting the libpq "logging"
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-logging.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-logging.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static bool pqTraceInit(PGconn *conn);
+static void pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+							PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceLogFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+	/*
+	 * Deallocate FE/BE message tracking memory.
+	 * If fields is allocated the initial size, we reuse it next time,
+	 * because it would be allocated same size and the size is not big.
+	 */
+	if (conn->fe_msg &&
+			conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() is failed, do noting. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->skipLogging = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->skipLogging = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer.
+ */
+static void
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+	}
+	else
+		fprintf(conn->Pfdebug, " ");
+}
+
+/*
+ * pqTraceForcelyCloseBeMsgLog:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+
+	return timestr;
+}
+
+/*
+ * pqTraceLogFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceLogFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+	const char	   *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+			* We delayed printing message type for special messages;
+			* they are complete now, so print them.
+			*/
+		if (conn->fe_msg->num_fields > 0)
+		{
+			int			message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+			memcpy(&result32, conn->outBuffer + message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+			else
+				message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqTraceLogMsgString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceLogMsgnchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceLogBeMsgByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceLogBeMsgByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			if (v != '\0')
+				fprintf(conn->Pfdebug, "%s\t",
+						pqGetProtocolMsgType((unsigned char) v,
+												MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqTraceLogBeMsgInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceLogBeMsgInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+
+/*
+ * pqTraceLogMsgString: output a null-terminated string to the log
+ */
+void
+pqTraceLogMsgString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqTraceLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceLogMsgnchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-logging.h b/src/interfaces/libpq/libpq-logging.h
new file mode 100644
index 0000000..60273cd
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-logging.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq logging.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-logging.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_LOGGING_H
+#define LIBPQ_LOGGING_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+	int			skipLogging;	/* state of whether to skip first byte1 and length */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+extern void pqTraceLogBeMsgByte1(PGconn *conn, char v);
+extern void pqTraceLogBeMsgInt(PGconn *conn, int v, int length);
+extern void pqTraceLogMsgString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceLogMsgnchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceLogFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_LOGGING_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1d540fe..09244d6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#112tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#111)
RE: libpq debug log

(48)
<synopsis>
void PQtrace(PGconn *conn, FILE *stream);
</synopsis>
</para>

+     <para>
+      Calls <function>PQtraceSetFlags</function> to output with or without a timestamp.
+     </para>
+
      <note>

Why is this necessary? Even if you decide to remove this change, can you share your idea on why you added this just in case?

(49)
+ Determine to output tracing with or without a timestamp to a debugging file stream.

The description should be the overall functionality of this function considering the future additions of flags. Something like:

Controls the tracing behavior of client/server communication.

+ then timestamp is not printed with each message. The default is output with timestamp.

The default value for flags argument (0) should be described here.

Also, it should be added that PQsetTraceFlags() can be called before or after the PQtrace() call.

(50)
I'd like you to consider skipLogging again, because it seems that such a variable is for the logging machinery to control state transitions internally in fe-logging.c.
What I'm concerned is that those who modify libpq have to be aware of skipLogging and would fail to handle it.

(51)

+typedef enum PGLogState
+{

This is libpq-logging.c internal type. It is not needed to be exposed.

I fixed it.

How did you fix it? The typedef is still in .h file. It should be in .h, shouldn't it? Houw about the other typedefs?

I won't comment on the other stuff on function naming.

Regards
Takayuki Tsunakawa

#113iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#112)
1 attachment(s)
RE: libpq debug log

Hi all,

I update patch to v18. It has been fixed in response to Tsunakawa san's review.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Friday, February 12, 2021 1:53 PM

(48)
<synopsis>
void PQtrace(PGconn *conn, FILE *stream);
</synopsis>
</para>

+     <para>
+      Calls <function>PQtraceSetFlags</function> to output with or
without a timestamp.
+     </para>
+
<note>

Why is this necessary? Even if you decide to remove this change, can you
share your idea on why you added this just in case?

In the previous patch, we described the PQTRACE_SUPPRESS_TIMESTAMPS flag in the PQtrace () part.
And I think PQtrace () and PQtraceSetFlag () are closely related functions.
It would be nice for the user to know both features before using the logging feature.
So I added a description of PQtraceSetFlag () to the paragraph of PQtrace ().
However, the PQtraceSetFlag () documentation is in the next paragraph of PQtrace (),
so users can easily notice the PQtraceSetFlag () function. So I removed it.

(49)
+ Determine to output tracing with or without a timestamp to a
debugging file stream.

The description should be the overall functionality of this function considering
the future additions of flags. Something like:

Controls the tracing behavior of client/server communication.

+ then timestamp is not printed with each message. The default is
output with timestamp.

The default value for flags argument (0) should be described here.

Also, it should be added that PQsetTraceFlags() can be called before or after
the PQtrace() call.

I fixed documentation like this;
If set to 0 (default),tracing will be output with timestamp.
This function should be called after calling <function>PQtrace</function>.

(50)
I'd like you to consider skipLogging again, because it seems that such a
variable is for the logging machinery to control state transitions internally in
fe-logging.c.
What I'm concerned is that those who modify libpq have to be aware of
skipLogging and would fail to handle it.

To implement inLogging, logging skip method has been modified to be completed within the libpq-logging.c file.
The protocol message consists of First Byte, Length, and Contents.
When the first byte and length arrive, if contents is not yet in the buffer,
the process of reading the first byte and length is repeated.
Reloading is necessary to parse the protocol message and proceed,
but we want to skip logging because it is not necessary to output the message which already log to file.

To do this, remember the value of inCursor in the inLogging variable and record how far logging has progressed.
In the pqTraceLogBeMsgByte1() and pqTraceLogBeMsgInt(),
the value of inCursor at that time is stored in inLogging after outputting the log to a file.
If inCursor is smaller than inLogging, it exits the log output function without doing anything.
If all log messages are output, initialize the inLogging.

Changed the pqTraceMaybeBreakLine() function to return whether a line break has occurred
in order to consolidate the process of recording the cursor in one place. If it break line,
inLogging is initialized and inCursor is not remembered inLogging.

Also, as Horiguchi san mentioned, I changed to update the value of inLogging
even in the function where the value of inCursor is changed (Ex. pqCheckInBufferSpace () and pqReadData ()).

(51)

+typedef enum PGLogState
+{

This is libpq-logging.c internal type. It is not needed to be exposed.

I fixed it.

How did you fix it? The typedef is still in .h file. It should be in .h, shouldn't
it? Houw about the other typedefs?

I didn’t move PGLogState to .c file because PGLogState is a member of be_msg which is referred from other files.

Regards,
Aya Iwata
Fujitsu

Attachments:

v18-0001-libpq-trace.patchapplication/octet-stream; name=v18-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 5e25f20..4513130 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5907,7 +5907,8 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
@@ -5927,6 +5928,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If (<literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),
+      then timestamp is not printed with each message. If set to 0 (default),tracing will be output with timestamp.
+      This function should be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677e..b703e0c 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-logging.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-logging.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir)/libpq-logging.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index db71fea..9bf2f0d 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4021,6 +4021,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6766,27 +6770,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..9147766 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-logging.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -85,7 +86,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+			pqTraceLogBeMsgByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceLogMsgString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceLogMsgnchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +269,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqTraceLogBeMsgInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -417,6 +410,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 					conn->inEnd - conn->inStart);
 			conn->inEnd -= conn->inStart;
 			conn->inCursor -= conn->inStart;
+			if (conn->Pfdebug)
+				conn->be_msg->inLogging -= conn->inStart;
 			conn->inStart = 0;
 		}
 	}
@@ -424,6 +419,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 	{
 		/* buffer is logically empty, reset it */
 		conn->inStart = conn->inCursor = conn->inEnd = 0;
+		if (conn->Pfdebug)
+			conn->be_msg->inLogging = 0;
 	}
 
 	/* Recheck whether we have enough space */
@@ -535,8 +532,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +568,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceLogFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -633,6 +628,8 @@ pqReadData(PGconn *conn)
 					conn->inEnd - conn->inStart);
 			conn->inEnd -= conn->inStart;
 			conn->inCursor -= conn->inStart;
+			if (conn->Pfdebug)
+				conn->be_msg->inLogging -= conn->inStart;
 			conn->inStart = 0;
 		}
 	}
@@ -640,6 +637,8 @@ pqReadData(PGconn *conn)
 	{
 		/* buffer is logically empty, reset it */
 		conn->inStart = conn->inCursor = conn->inEnd = 0;
+		if (conn->Pfdebug)
+			conn->be_msg->inLogging = 0;
 	}
 
 	/*
@@ -1011,11 +1010,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..efaf310 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -283,6 +283,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +360,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +372,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +402,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +466,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1635,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index effe0cc..cdbaa2a 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ce36aab..5944400 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -22,6 +22,7 @@
 
 /* We assume libpq-fe.h has already been included. */
 #include "libpq-events.h"
+#include "libpq-logging.h"
 
 #include <time.h>
 #ifndef WIN32
@@ -376,6 +377,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-logging.c b/src/interfaces/libpq/libpq-logging.c
new file mode 100644
index 0000000..84ae43c
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.c
@@ -0,0 +1,637 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-logging.c
+ *	  functions for supporting the libpq "logging"
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-logging.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-logging.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static bool pqTraceInit(PGconn *conn);
+static bool pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+							PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceLogFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+	/*
+	 * Deallocate FE/BE message tracking memory.
+	 * If fields is allocated the initial size, we reuse it next time,
+	 * because it would be allocated same size and the size is not big.
+	 */
+	if (conn->fe_msg &&
+			conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() is failed, do noting. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->inLogging = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->inLogging = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer. If print break line, return 1.
+ */
+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+		return 1;
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, " ");
+		return 0;
+	}
+}
+
+/*
+ * pqTraceForcelyCloseBeMsgLog:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+
+	return timestr;
+}
+
+/*
+ * pqTraceLogFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceLogFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+	const char	   *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+			* We delayed printing message type for special messages;
+			* they are complete now, so print them.
+			*/
+		if (conn->fe_msg->num_fields > 0)
+		{
+			int			message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+			memcpy(&result32, conn->outBuffer + message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+			else
+				message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqTraceLogMsgString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceLogMsgnchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceLogBeMsgByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceLogBeMsgByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->inLogging >= conn->inCursor)
+		return;
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			fprintf(conn->Pfdebug, "%s\t",
+					pqGetProtocolMsgType((unsigned char) v,
+											MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			logfinish = pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->inLogging = conn->inCursor;
+}
+
+/*
+ * pqTraceLogBeMsgInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceLogBeMsgInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->inLogging >= conn->inCursor)
+		return;
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			logfinish = pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->inLogging = conn->inCursor;
+}
+
+
+/*
+ * pqTraceLogMsgString: output a null-terminated string to the log
+ */
+void
+pqTraceLogMsgString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqTraceLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceLogMsgnchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-logging.h b/src/interfaces/libpq/libpq-logging.h
new file mode 100644
index 0000000..9720f78
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-logging.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq logging.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-logging.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_LOGGING_H
+#define LIBPQ_LOGGING_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+	int			inLogging;		/* next byte of logging */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+extern void pqTraceLogBeMsgByte1(PGconn *conn, char v);
+extern void pqTraceLogBeMsgInt(PGconn *conn, int v, int length);
+extern void pqTraceLogMsgString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceLogMsgnchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceLogFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_LOGGING_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..fad9029 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#114tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#113)
RE: libpq debug log

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

I update patch to v18. It has been fixed in response to Tsunakawa san's review.

(52)
+ of tracing. If (<literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),

() can be removed?

(53)
+ int inLogging; /* next byte of logging */

I understood this variable contains a position or offset from some place. Then, isn't it better to call it LogPos, LogOffset, or LogCursor? I don't think you need to prepend "In" as in InCursor, because you don't have to distinguish between input and output unlike InCursor in PGconn.

(54)
Can't you restrict the use of the above InLogging in fe-logging.c? How about making InLogging mean the offset from InStart, i.e. the offset from the start of a message?

Regards
Takayuki Tsunakawa

#115iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#114)
1 attachment(s)
RE: libpq debug log

Hi Tsunakawa san,

I update patch to v19.

-----Original Message-----
From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Monday, February 22, 2021 1:30 PM
(52)
+      of tracing.  If (<literal>flags</literal> contains
<literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>),

() can be removed?

Yes, I removed ().

(53)
+ int inLogging; /* next byte of
logging */

I understood this variable contains a position or offset from some place. Then,
isn't it better to call it LogPos, LogOffset, or LogCursor? I don't think you
need to prepend "In" as in InCursor, because you don't have to distinguish
between input and output unlike InCursor in PGconn.

Yes, "In" is from inCursor. I change inLogging to LogCursor because this name easier to understand
the meaning of this parameter.

(54)
Can't you restrict the use of the above InLogging in fe-logging.c? How about
making InLogging mean the offset from InStart, i.e. the offset from the start of
a message?

I fixed it.
Remove inLogging code from pqCheckInBufferSpace() and pqReadData().
And LogCursor remember the difference between conn->inCursor and conn->inStart.

And 3 bugs I noted February 2nd email are all fixed.

1. fix 3 bugs
1.1 -1 output in "Query" message
1.2 two message output in "ReadyForQuery" message
1.3 "StartupMessage" output as " UnknownMessage "

If you want to know how to fix there bugs, please see my email;

Sent: Monday, February 8, 2021 11:58 PM

Regards,
Aya Iwata
Fujitsu

Attachments:

v19-0001-libpq-trace.patchapplication/octet-stream; name=v19-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 5e25f20..3a1e368 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5907,7 +5907,8 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
@@ -5927,6 +5928,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then timestamp is not printed with each message. If set to 0 (default),tracing will be output with timestamp.
+      This function should be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677e..b703e0c 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-logging.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-logging.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir)/libpq-logging.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index db71fea..9bf2f0d 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4021,6 +4021,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6766,27 +6770,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..236996d 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-logging.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -85,7 +86,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+			pqTraceLogBeMsgByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceLogMsgString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceLogMsgnchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len,
+						  MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +269,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqTraceLogBeMsgInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +528,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +564,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceLogFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -1011,11 +1002,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..efaf310 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -283,6 +283,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +360,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +372,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +402,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +466,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1635,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index effe0cc..cdbaa2a 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ce36aab..5944400 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -22,6 +22,7 @@
 
 /* We assume libpq-fe.h has already been included. */
 #include "libpq-events.h"
+#include "libpq-logging.h"
 
 #include <time.h>
 #ifndef WIN32
@@ -376,6 +377,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-logging.c b/src/interfaces/libpq/libpq-logging.c
new file mode 100644
index 0000000..68bc1f2
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.c
@@ -0,0 +1,637 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-logging.c
+ *	  functions for supporting the libpq "logging"
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-logging.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-logging.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0,	0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static bool pqTraceInit(PGconn *conn);
+static bool pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+							PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceLogFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+	/*
+	 * Deallocate FE/BE message tracking memory.
+	 * If fields is allocated the initial size, we reuse it next time,
+	 * because it would be allocated same size and the size is not big.
+	 */
+	if (conn->fe_msg &&
+			conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() is failed, do noting. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->LogCursor = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->LogCursor = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer. If print break line, return 1.
+ */
+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+		return 1;
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, " ");
+		return 0;
+	}
+}
+
+/*
+ * pqTraceForcelyCloseBeMsgLog:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with milliseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+
+	return timestr;
+}
+
+/*
+ * pqTraceLogFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceLogFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+	const char	   *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+			* We delayed printing message type for special messages;
+			* they are complete now, so print them.
+			*/
+		if (conn->fe_msg->num_fields > 0)
+		{
+			int			message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+			memcpy(&result32, conn->outBuffer + message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+			else
+				message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqTraceLogMsgString(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceLogMsgnchar(conn, conn->outBuffer + message_addr,
+								  length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceLogBeMsgByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceLogBeMsgByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char		*timestr_p = "";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->LogCursor >= conn->inCursor - conn->inStart)
+		return;
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			fprintf(conn->Pfdebug, "%s\t",
+					pqGetProtocolMsgType((unsigned char) v,
+											MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+			/*
+				* Show non-printable data in hex format, including the
+				* terminating \0 that completes ErrorResponse and
+				* NoticeResponse messages.
+				*/
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			logfinish = pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->LogCursor = conn->inCursor - conn->inStart;
+}
+
+/*
+ * pqTraceLogBeMsgInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceLogBeMsgInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->LogCursor >= conn->inCursor - conn->inStart)
+		return;
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			logfinish = pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->LogCursor = conn->inCursor - conn->inStart;
+}
+
+
+/*
+ * pqTraceLogMsgString: output a null-terminated string to the log
+ */
+void
+pqTraceLogMsgString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				pin;
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;	/* XXX ??? */
+	}
+
+	for (pin = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + pin, 1, i - pin, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			pin = i + 1;
+		}
+	}
+	if (pin < length)
+		fwrite(v + pin, 1, length - pin, conn->Pfdebug);
+}
+
+/*
+ * pqTraceLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceLogMsgnchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-logging.h b/src/interfaces/libpq/libpq-logging.h
new file mode 100644
index 0000000..16e5934
--- /dev/null
+++ b/src/interfaces/libpq/libpq-logging.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-logging.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq logging.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-logging.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_LOGGING_H
+#define LIBPQ_LOGGING_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+	int			LogCursor;		/* next byte of logging */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+extern void pqTraceLogBeMsgByte1(PGconn *conn, char v);
+extern void pqTraceLogBeMsgInt(PGconn *conn, int v, int length);
+extern void pqTraceLogMsgString(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceLogMsgnchar(PGconn *conn, const char *v, int length,
+							  PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceLogFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_LOGGING_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..fad9029 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#116tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#115)
RE: libpq debug log

Alvaro-san, Iwata-san,

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

I update patch to v19.

...

And 3 bugs I noted February 2nd email are all fixed.

1. fix 3 bugs
1.1 -1 output in "Query" message
1.2 two message output in "ReadyForQuery" message
1.3 "StartupMessage" output as " UnknownMessage "

That's nice. And I've confirmed all review comments have been reflected.

Alvaro-san,
We're sorry to be late. I appreciate your patience and follow-up. Could you do the final check for commit?

Regards
Takayuki Tsunakawa

#117alvherre@alvh.no-ip.org
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#115)
1 attachment(s)
Re: libpq debug log

I'll give this another look tomorrow, but I wanted to pass along that I
prefer libpq-trace.{c,h} instead of libpq-logging. I also renamed
variable "pin" and pgindented. I don't have any major reservations
about this patch now, so I'll mark it ready-for-committer in case
somebody else wants to have a look before push.

(Not sure about the use of the word "forcely")

--
�lvaro Herrera 39�49'30"S 73�17'W

Attachments:

v20-0001-Improve-libpq-tracing-capabilities.patchtext/x-diff; charset=us-asciiDownload
From da0faf61ceb8e9cbc5894ca1e43b6f7fc477a835 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Tue, 23 Feb 2021 22:17:51 -0300
Subject: [PATCH v20] Improve libpq tracing capabilities

Author: Aya Iwata
---
 doc/src/sgml/libpq.sgml             |  25 +-
 src/interfaces/libpq/Makefile       |   3 +
 src/interfaces/libpq/exports.txt    |   1 +
 src/interfaces/libpq/fe-connect.c   |  25 +-
 src/interfaces/libpq/fe-exec.c      |   4 -
 src/interfaces/libpq/fe-misc.c      |  59 ++-
 src/interfaces/libpq/fe-protocol3.c |  18 +
 src/interfaces/libpq/libpq-fe.h     |   2 +
 src/interfaces/libpq/libpq-int.h    |   6 +
 src/interfaces/libpq/libpq-trace.c  | 638 ++++++++++++++++++++++++++++
 src/interfaces/libpq/libpq-trace.h  |  95 +++++
 src/tools/pgindent/typedefs.list    |   6 +
 12 files changed, 823 insertions(+), 59 deletions(-)
 create mode 100644 src/interfaces/libpq/libpq-trace.c
 create mode 100644 src/interfaces/libpq/libpq-trace.h

diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 5e25f20843..3a1e368e0b 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5907,7 +5907,8 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
@@ -5927,6 +5928,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then timestamp is not printed with each message. If set to 0 (default),tracing will be output with timestamp.
+      This function should be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677eaf9..cdb65b48af 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir)/libpq-trace.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90481..09f11114a6 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index db71fea169..9bf2f0d0b6 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4021,6 +4021,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6766,27 +6770,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753387..e8503aabc7 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6acd89..e36a2348d9 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -85,7 +86,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqTraceLogBeMsgByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceLogMsgString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceLogMsgnchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len,
+						   MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +269,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqTraceLogBeMsgInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +528,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +564,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceLogFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -1011,11 +1002,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d69d2..90ce13eade 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -283,6 +283,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +360,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +372,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +402,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+					/* Terminate a half-finished logging message */
+					if (conn->Pfdebug)
+						pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +466,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1635,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index effe0ccf85..cdbaa2a987 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ce36aabd25..f0002ec51f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -22,6 +22,7 @@
 
 /* We assume libpq-fe.h has already been included. */
 #include "libpq-events.h"
+#include "libpq-trace.h"
 
 #include <time.h>
 #ifndef WIN32
@@ -376,6 +377,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000000..821380bcf1
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,638 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-trace.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static bool pqTraceInit(PGconn *conn);
+static bool pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+										PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceLogFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	/*
+	 * Deallocate FE/BE message tracking memory. If the fe_msg allocation is
+	 * of the initial size, reuse it next time.
+	 */
+	if (conn->fe_msg &&
+		conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() is failed, do noting. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->LogCursor = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->LogCursor = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break and reset the buffer. If print break line, return 1.
+ */
+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+		return 1;
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, " ");
+		return 0;
+	}
+}
+
+/*
+ * pqTraceForcelyCloseBeMsgLog:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+
+	return timestr;
+}
+
+/*
+ * pqTraceLogFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceLogFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char	   *timestr_p = "";
+	const char *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+		 * We delayed printing message type for special messages; they are
+		 * complete now, so print them.
+		 */
+		if (conn->fe_msg->num_fields > 0)
+		{
+			int			message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+			memcpy(&result32, conn->outBuffer + message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+		else
+			message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqTraceLogMsgString(conn, conn->outBuffer + message_addr,
+									length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceLogMsgnchar(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceLogBeMsgByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceLogBeMsgByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char	   *timestr_p = "";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->LogCursor >= conn->inCursor - conn->inStart)
+		return;
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			fprintf(conn->Pfdebug, "%s\t",
+					pqGetProtocolMsgType((unsigned char) v,
+										 MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+
+			/*
+			 * Show non-printable data in hex format, including the
+			 * terminating \0 that completes ErrorResponse and NoticeResponse
+			 * messages.
+			 */
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			logfinish = pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->LogCursor = conn->inCursor - conn->inStart;
+}
+
+/*
+ * pqTraceLogBeMsgInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceLogBeMsgInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->LogCursor >= conn->inCursor - conn->inStart)
+		return;
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			logfinish = pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->LogCursor = conn->inCursor - conn->inStart;
+}
+
+
+/*
+ * pqTraceLogMsgString: output a null-terminated string to the log
+ */
+void
+pqTraceLogMsgString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, conn->Pfdebug);
+}
+
+/*
+ * pqTraceLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceLogMsgnchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-trace.h b/src/interfaces/libpq/libpq-trace.h
new file mode 100644
index 0000000000..ff3be37b77
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-trace.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq protocol tracing.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-trace.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_TRACE_H
+#define LIBPQ_TRACE_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+	int			LogCursor;		/* next byte of logging */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+extern void pqTraceLogBeMsgByte1(PGconn *conn, char v);
+extern void pqTraceLogBeMsgInt(PGconn *conn, int v, int length);
+extern void pqTraceLogMsgString(PGconn *conn, const char *v, int length,
+								PGCommSource commsource);
+extern void pqTraceLogMsgnchar(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceLogFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_TRACE_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3adb3..fad9029ac5 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
-- 
2.20.1

#118tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: alvherre@alvh.no-ip.org (#117)
RE: libpq debug log

From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>

I'll give this another look tomorrow, but I wanted to pass along that I prefer
libpq-trace.{c,h} instead of libpq-logging. I also renamed variable "pin" and
pgindented.

Ah, you're right, because the function names are PQtrace() and PQuntrace().

I don't have any major reservations about this patch now, so I'll
mark it ready-for-committer in case somebody else wants to have a look before
push.

Actually, I had marked it as RFC just before I sent my previous mail. Anyway, I'm relieved to hear that. We hope this will be committed in PG 14.

(Not sure about the use of the word "forcely")

Hmm, "forcedly" is not in my English dictionary. It should probably be "forcibly", or it can be removed, I think.

Regards
Takayuki Tsunakawa

#119k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: alvherre@alvh.no-ip.org (#117)
RE: libpq debug log

From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
I'll give this another look tomorrow, but I wanted to pass along that I prefer
libpq-trace.{c,h} instead of libpq-logging. I also renamed variable "pin" and
pgindented. I don't have any major reservations about this patch now, so I'll
mark it ready-for-committer in case somebody else wants to have a look
before push.

(Not sure about the use of the word "forcely")

Hi Iwata-san and Alvaro-san,

Thank you for updating the patch.
Although it's already set as "Ready for Committer", I just found minor parts
that need to be fixed.

(1) Doc: PQtraceSetFlags
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>, 
+      then timestamp is not printed with each message. If set to 0 (default),tracing will be output with timestamp.
+      This function should be called after calling <function>PQtrace</function>.

Missing space. And can be paraphrased to:
"If set to 0 (default), tracing with timestamp is printed."

(2)
+ * pqTraceMaybeBreakLine:
+ *             Check whether the backend message is complete. If so, print a line
+ *             break and reset the buffer. If print break line, return 1.

The 2nd & last sentence can be combined to
"If so, print a line break, reset the buffer, and return 1."

(3) +PQtraceSetFlags(PGconn *conn, int flags)
+ /* If PQtrace() is failed, do noting. */

"If PQtrace() failed, do nothing."

(4)

(Not sure about the use of the word "forcely")

I think it's not necessary.

Also, I tested the flag to not print timestamp. For example,
PQtrace(conn, trace_file);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);

And it did not print the timestamp. So it worked.
It also passed all the regression tests. (although PQtrace() is not tested in existing libpq tests).

Regards,
Kirk Jamison

#120iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: k.jamison@fujitsu.com (#119)
1 attachment(s)
RE: libpq debug log

Hi Kirk san,

Thank you for your review. I update patch to v21.

-----Original Message-----
From: Jamison, Kirk/ジャミソン カーク <k.jamison@fujitsu.com>
Sent: Wednesday, February 24, 2021 1:04 PM

(1) Doc: PQtraceSetFlags
+      <literal>flags</literal> contains flag bits describing the operating
mode
+      of tracing.  If <literal>flags</literal> contains
<literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then timestamp is not printed with each message. If set to 0
(default),tracing will be output with timestamp.
+      This function should be called after calling
<function>PQtrace</function>.

Missing space. And can be paraphrased to:
"If set to 0 (default), tracing with timestamp is printed."

I fixed this documentation as you suggested.

(2)
+ * pqTraceMaybeBreakLine:
+ *             Check whether the backend message is complete. If so, print
a line
+ *             break and reset the buffer. If print break line, return 1.

The 2nd & last sentence can be combined to "If so, print a line break, reset
the buffer, and return 1."

I fixed it because it is more natural than previous explanation.

(3) +PQtraceSetFlags(PGconn *conn, int flags)
+ /* If PQtrace() is failed, do noting. */

"If PQtrace() failed, do nothing."

I fixed it.

(4)

(Not sure about the use of the word "forcely")

I think it's not necessary.

Sure.

Also, I tested the flag to not print timestamp. For example,
PQtrace(conn, trace_file);
PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);

And it did not print the timestamp. So it worked.
It also passed all the regression tests. (although PQtrace() is not tested in
existing libpq tests).

Thank you for your test.

Regards,
Aya Iwata
Fujitsu

Attachments:

v21-0001-libpq-trace.patchapplication/octet-stream; name=v21-0001-libpq-trace.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 5e25f20..ff36ae6 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5907,7 +5907,8 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file stream.
+      (Details of tracing contents appear in <xref linkend="protocol-message-formats"/>).
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
@@ -5927,6 +5928,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.  If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then timestamp is not printed with each message. If set to 0 (default), tracing with timestamp is printed.
+      This function should be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677e..cdb65b4 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir)/libpq-trace.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index db71fea..9bf2f0d 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4021,6 +4021,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6766,27 +6770,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753..e8503aa 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6ac..e36a234 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -85,7 +86,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqTraceLogBeMsgByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceLogMsgString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceLogMsgnchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceLogMsgnchar(conn, conn->inBuffer + conn->inCursor, len,
+						   MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +269,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqTraceLogBeMsgInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +528,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +564,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceLogFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -1011,11 +1002,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d6..90ce13e 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -283,6 +283,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +360,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +372,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +402,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+					/* Terminate a half-finished logging message */
+					if (conn->Pfdebug)
+						pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +466,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1635,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForcelyCloseBeMsgLog(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index effe0cc..cdbaa2a 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ce36aab..f0002ec 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -22,6 +22,7 @@
 
 /* We assume libpq-fe.h has already been included. */
 #include "libpq-events.h"
+#include "libpq-trace.h"
 
 #include <time.h>
 #ifndef WIN32
@@ -376,6 +377,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..ec8cbe8
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,638 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-trace.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static bool pqTraceInit(PGconn *conn);
+static bool pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqLogBinaryMsg(PGconn *conn, const char *v, int length,
+						   PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+										PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceLogFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	/*
+	 * Deallocate FE/BE message tracking memory. If the fe_msg allocation is
+	 * of the initial size, reuse it next time.
+	 */
+	if (conn->fe_msg &&
+		conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->LogCursor = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->LogCursor = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break, reset the buffer, and return 1.
+ */
+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+		return 1;
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, " ");
+		return 0;
+	}
+}
+
+/*
+ * pqTraceForcelyCloseBeMsgLog:
+ * 		If message is not completed, print a line break and reset.
+ */
+void
+pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset_in_buffer = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.  Used for PQtrace() purposes.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static char *
+pqLogFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+
+	return timestr;
+}
+
+/*
+ * pqTraceLogFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceLogFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char	   *timestr_p = "";
+	const char *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+		 * We delayed printing message type for special messages; they are
+		 * complete now, so print them.
+		 */
+		if (conn->fe_msg->num_fields > 0)
+		{
+			int			message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->fe_msg->fields[0].offset_in_buffer;
+			memcpy(&result32, conn->outBuffer + message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+		else
+			message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr_p,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		int			message_addr;
+		int			length;
+		char		v;
+
+		message_addr = conn->fe_msg->fields[i].offset_in_buffer;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				v = *(conn->outBuffer + message_addr);
+
+				if (isprint(v))
+					fprintf(conn->Pfdebug, "%c", v);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", v);
+				break;
+
+			case LOG_STRING:
+				pqTraceLogMsgString(conn, conn->outBuffer + message_addr,
+									length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceLogMsgnchar(conn, conn->outBuffer + message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, conn->outBuffer + message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, conn->outBuffer + message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceLogBeMsgByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceLogBeMsgByte1(PGconn *conn, char v)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char	   *timestr_p = "";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->LogCursor >= conn->inCursor - conn->inStart)
+		return;
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+				timestr_p = pqLogFormatTimestamp(timestr, sizeof(timestr));
+			fprintf(conn->Pfdebug, "%s\t<\t", timestr_p);
+			fprintf(conn->Pfdebug, "%s\t",
+					pqGetProtocolMsgType((unsigned char) v,
+										 MSGDIR_FROM_BACKEND));
+			/* Next, log the message length */
+			conn->be_msg->state = LOG_LENGTH;
+			conn->be_msg->command = v;
+			break;
+
+		case LOG_CONTENTS:
+
+			/*
+			 * Show non-printable data in hex format, including the
+			 * terminating \0 that completes ErrorResponse and NoticeResponse
+			 * messages.
+			 */
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+			logfinish = pqTraceMaybeBreakLine(1, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->LogCursor = conn->inCursor - conn->inStart;
+}
+
+/*
+ * pqTraceLogBeMsgInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceLogBeMsgInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->LogCursor >= conn->inCursor - conn->inStart)
+		return;
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			logfinish = pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->LogCursor = conn->inCursor - conn->inStart;
+}
+
+
+/*
+ * pqTraceLogMsgString: output a null-terminated string to the log
+ */
+void
+pqTraceLogMsgString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqLogBinaryMsg: output a string possibly consisting of non-printable
+ * characters. Hex representation is used for such chars; others are
+ * printed normally.
+ */
+static void
+pqLogBinaryMsg(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, conn->Pfdebug);
+}
+
+/*
+ * pqTraceLogMsgnchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceLogMsgnchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqLogBinaryMsg(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-trace.h b/src/interfaces/libpq/libpq-trace.h
new file mode 100644
index 0000000..ff3be37
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-trace.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq protocol tracing.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-trace.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_TRACE_H
+#define LIBPQ_TRACE_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the first byte identifying the
+								 * protocol message type */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	int			length;			/* protocol message length */
+	char		command;		/* first one byte of protocol message */
+	int			LogCursor;		/* next byte of logging */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset_in_buffer;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+extern void pqTraceLogBeMsgByte1(PGconn *conn, char v);
+extern void pqTraceLogBeMsgInt(PGconn *conn, int v, int length);
+extern void pqTraceLogMsgString(PGconn *conn, const char *v, int length,
+								PGCommSource commsource);
+extern void pqTraceLogMsgnchar(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceLogFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForcelyCloseBeMsgLog(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_TRACE_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..fad9029 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#121Álvaro Herrera
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#120)
1 attachment(s)
Re: libpq debug log

I tweaked this code a little bit more. I didn't like that libpq-trace.h was exposing all the internal details of the API to the world; and I liked it even less that libpq-int.h had to include the file, exposing everything everywhere. I added some forward struct declarations in libpq-int.h to avoid including libpq-trace.h in it; and I also moved a few of the definitions from libpq-trace.h to the .c file. We didn't really need them to be exposed outside the .c file. In the makefile, libpq-trace.h was being installed in the wrong place -- I changed so that it goes to includedir_internal, like libpq-int.h.

The function pqLogFormatTimestamp was returning a pointer to where it printed things. But that was pointless, because the buffer is caller supplied, so the caller knows full well where the output is. There's no gain in functionality, so I made it return void.

I added a description of the output format to the documentation.

I renamed a lot of functions, so that it looks more like a consistent API. pqTraceMaybeBreakLine() says it returns bool, but it was using "return 1/0" rather than actual bools. That's poor style.

The only thing we're missing here is some coverage. Right now, nothing in the source tree calls this at all, so if we break it, we'll never know. I think we should introduce src/test/modules/test_pqtrace or something like that.

v22 attached.

Attachments:

v22-0001-Improve-libpq-tracing-capabilities.patchtext/x-patch; name=v22-0001-Improve-libpq-tracing-capabilities.patchDownload
From 918a674b93938e23d6982273e500d6caa59b6079 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Thu, 25 Feb 2021 14:32:00 -0300
Subject: [PATCH v22] Improve libpq tracing capabilities

Author: Aya Iwata
---
 doc/src/sgml/libpq.sgml             |  40 +-
 src/interfaces/libpq/Makefile       |   3 +
 src/interfaces/libpq/exports.txt    |   1 +
 src/interfaces/libpq/fe-connect.c   |  25 +-
 src/interfaces/libpq/fe-exec.c      |   4 -
 src/interfaces/libpq/fe-misc.c      |  59 ++-
 src/interfaces/libpq/fe-protocol3.c |  19 +
 src/interfaces/libpq/libpq-fe.h     |   2 +
 src/interfaces/libpq/libpq-int.h    |   9 +
 src/interfaces/libpq/libpq-trace.c  | 689 ++++++++++++++++++++++++++++
 src/interfaces/libpq/libpq-trace.h  |  58 +++
 src/tools/pgindent/typedefs.list    |   6 +
 12 files changed, 856 insertions(+), 59 deletions(-)
 create mode 100644 src/interfaces/libpq/libpq-trace.c
 create mode 100644 src/interfaces/libpq/libpq-trace.h

diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 5e25f20843..adec5b4b82 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5907,12 +5907,28 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
+      Only available when protocol version 3 or higher is used.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message type, message length, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      For numerical quantities, 16-bit values are preceded with <literal>#</literal>
+      and 32-bit values are printed without a prefix.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -5927,6 +5943,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index f74677eaf9..5a690be83f 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -42,6 +42,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -114,6 +115,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -127,6 +129,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir_internal)/libpq-trace.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90481..09f11114a6 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index db71fea169..9bf2f0d0b6 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4021,6 +4021,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6766,27 +6770,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index e730753387..e8503aabc7 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index 2bfb6acd89..49067daebd 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -85,7 +86,7 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqTraceOutputBeByte1(conn, *result);
 
 	return 0;
 }
@@ -101,7 +102,7 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
@@ -139,8 +140,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceOutputString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -164,11 +164,13 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +191,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceOutputNchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +211,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceOutputNchar(conn, conn->inBuffer + conn->inCursor, len,
+						   MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +229,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +269,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqTraceOutputBeInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +284,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +308,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -535,8 +528,7 @@ pqPutMsgStart(char msg_type, bool force_len, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -572,15 +564,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceOutputFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -1011,11 +1002,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index e4ee9d69d2..76a65b0302 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -28,6 +28,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "port/pg_bswap.h"
 
@@ -283,6 +284,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForciblyCloseBeMsg(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +361,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForciblyCloseBeMsg(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +373,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForciblyCloseBeMsg(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +403,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+					/* Terminate a half-finished logging message */
+					if (conn->Pfdebug)
+						pqTraceForciblyCloseBeMsg(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +467,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForciblyCloseBeMsg(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1636,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForciblyCloseBeMsg(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index effe0ccf85..cdbaa2a987 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -363,7 +363,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ce36aabd25..9ff6e5569d 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -317,6 +317,10 @@ typedef struct pg_conn_host
 								 * found in password file. */
 } pg_conn_host;
 
+/* forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 /*
  * PGconn stores all the state data associated with a single connection
  * to a backend.
@@ -376,6 +380,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000000..f774ad062a
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,689 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-trace.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the message type byte */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+/*
+ * Messages received from backend are printed to the trace file as soon as
+ * processed from the server.
+ */
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	char		command;		/* protocol message type */
+	int			length;			/* protocol message length */
+	int			logCursor;		/* next byte of logging */
+} pqBackendMessage;
+
+/*
+ * Frontend messages are accumulated as individual fields that each is a
+ * pointer into conn->outBuffer.
+ */
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+
+static bool pqTraceInit(PGconn *conn);
+static bool pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqTraceOutputBinary(PGconn *conn, const char *v, int length,
+								PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+										PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceOutputFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	/*
+	 * Deallocate FE/BE message tracking memory. If the fe_msg allocation is
+	 * of the initial size, reuse it next time.
+	 */
+	if (conn->fe_msg &&
+		conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->logCursor = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->logCursor = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break, reset the buffer, and return true.  If there's still more
+ *		in the backend message, print a blank and return false.
+ */
+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	Assert(conn->be_msg->length > 0);
+
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+		return true;
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, " ");
+		return false;
+	}
+}
+
+/*
+ * pqTraceForciblyCloseBeMsg
+ * 		Print a newline and reset backend message state.
+ *
+ * To be used when dealing with various errors.
+ */
+void
+pqTraceForciblyCloseBeMsg(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+			return;
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ * pqTraceOutputFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceOutputFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	const char *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+		 * We delayed printing message type for special messages; they are
+		 * complete now, so print them.
+		 */
+		if (conn->fe_msg->num_fields > 0)
+		{
+			char	   *message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->outBuffer + conn->fe_msg->fields[0].offset;
+			memcpy(&result32, message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+		else
+			message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		char	   *message_addr;
+		int			length;
+
+		message_addr = conn->outBuffer + conn->fe_msg->fields[i].offset;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				if (isprint(*message_addr))
+					fprintf(conn->Pfdebug, "%c", *message_addr);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", *message_addr);
+				break;
+
+			case LOG_STRING:
+				pqTraceOutputString(conn, message_addr,
+									length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceOutputNchar(conn, message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceOutputBeByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceOutputBeByte1(PGconn *conn, char v)
+{
+	if (conn->be_msg->logCursor >= conn->inCursor - conn->inStart)
+		return;
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			{
+				char		timestr[FORMATTED_TS_LEN];
+
+				if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+					pqTraceFormatTimestamp(timestr, sizeof(timestr));
+				else
+					timestr[0] = '\0';
+				fprintf(conn->Pfdebug, "%s\t<\t%s\t", timestr,
+						pqGetProtocolMsgType((unsigned char) v,
+											 MSGDIR_FROM_BACKEND));
+				/* Next, log the message length */
+				conn->be_msg->state = LOG_LENGTH;
+				conn->be_msg->command = v;
+				break;
+			}
+
+		case LOG_CONTENTS:
+
+			/*
+			 * Show non-printable data in hex format, including the
+			 * terminating \0 that completes ErrorResponse and NoticeResponse
+			 * messages.
+			 */
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+
+			if (!pqTraceMaybeBreakLine(1, conn))
+				conn->be_msg->logCursor = conn->inCursor - conn->inStart;
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqTraceOutputBeInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceOutputBeInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	bool		logfinish = 0;
+
+	if (conn->be_msg->logCursor >= conn->inCursor - conn->inStart)
+		return;
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v - length;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			logfinish = pqTraceMaybeBreakLine(0, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->logCursor = conn->inCursor - conn->inStart;
+}
+
+
+/*
+ * pqTraceOutputString: output a null-terminated string to the log
+ */
+void
+pqTraceOutputString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqTraceOutputBinary: output a string possibly consisting of
+ * non-printable characters. Hex representation is used for such
+ * chars; others are printed normally.
+ *
+ * XXX this probably doesn't do a great job with multibyte chars, but then we
+ * don't know what is text and what encoding it'd be in.
+ */
+static void
+pqTraceOutputBinary(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, conn->Pfdebug);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceOutputNchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceOutputBinary(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	if (commsource == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-trace.h b/src/interfaces/libpq/libpq-trace.h
new file mode 100644
index 0000000000..f1e1e6ffb2
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-trace.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq protocol tracing.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-trace.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_TRACE_H
+#define LIBPQ_TRACE_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+extern void pqTraceOutputBeByte1(PGconn *conn, char v);
+extern void pqTraceOutputBeInt(PGconn *conn, int v, int length);
+extern void pqTraceOutputString(PGconn *conn, const char *v, int length,
+								PGCommSource commsource);
+extern void pqTraceOutputNchar(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceOutputFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForciblyCloseBeMsg(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_TRACE_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3adb3..fad9029ac5 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3273,6 +3276,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
-- 
2.20.1

#122Álvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Álvaro Herrera (#121)
Re: libpq debug log

It appears that something is still wrong. I applied lipq pipeline v27 from [1]/messages/by-id/20210216231350.GA1629@alvherre.pgsql and ran src/test/modules/test_libpq/pipeline singlerow, after patching it to do PQtrace() after PQconn(). Below is the output I get from that. The noteworthy point is that "ParseComplete" messages appear multiple times for some reason ... but that's quite odd, because if I look at the network traffic with Wireshark I certainly do not see the ParseComplete message being sent three times.

[1]: /messages/by-id/20210216231350.GA1629@alvherre.pgsql

Parse 38 "" "SELECT generate_series(42, $1)" #0
Bind 20 "" "" #0 #1 2 '44' #1 #0
Describe 6 P ""
Execute 9 "" 0
Parse 38 "" "SELECT generate_series(42, $1)" #0
Bind 20 "" "" #0 #1 2 '45' #1 #0
Describe 6 P ""
Execute 9 "" 0
Parse 38 "" "SELECT generate_series(42, $1)" #0
Bind 20 "" "" #0 #1 2 '46' #1 #0
Describe 6 P ""
Execute 9 "" 0
Sync 4

< ParseComplete 4
< BindComplete 4
< RowDescription 40 #1 "generate_series" 0 #0 23 #4 -1 #0
< DataRow 12 #1 2 '42'
< DataRow 12 #1 2 '43'
< DataRow 12 #1 2 '44'
< CommandComplete 13 "SELECT 3"
< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4
< RowDescription 40 #1 "generate_series" 0 #0 23 #4 -1 #0
< DataRow 12 #1 2 '42'
< DataRow 12 #1 2 '43'
< DataRow 12 #1 2 '44'
< DataRow 12 #1 2 '45'
< CommandComplete 13 "SELECT 4"
< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4
< RowDescription 40 #1 "generate_series" 0 #0 23 #4 -1 #0
< DataRow 12 #1 2 '42'
< DataRow 12 #1 2 '43'
< DataRow 12 #1 2 '44'
< DataRow 12 #1 2 '45'
< DataRow 12 #1 2 '46'
< CommandComplete 13 "SELECT 5"
< ReadyForQuery 5 I

Show quoted text

Terminate 4

#123Álvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Álvaro Herrera (#121)
Re: libpq debug log

This assert I added?

+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	Assert(conn->be_msg->length > 0);

It breaks. Maybe there's a good reason, maybe there ain't, but in the
meantime I just removed it.

--
�lvaro Herrera Valdivia, Chile
"Para tener m�s hay que desear menos"

#124iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Álvaro Herrera (#121)
RE: libpq debug log

Alvaro san,

Thank you very much for your updating and organizing this patch.

It appears that something is still wrong. I applied lipq pipeline v27 from [1] and ran src/test/modules/test_libpq/pipeline singlerow, after patching it to do PQtrace() after PQconn(). Below is the output I get from that. The noteworthy point is that "ParseComplete" messages appear multiple times for some reason ... but that's quite odd, because if I look at the network traffic with Wireshark I certainly do not see the ParseComplete message being sent three times.

I will search this cause. Please wait a minuets.
I thought I could avoid multiple such outputs by using conn->LogCursor…

Regards,
Aya Iwata
Fujitsu

From: Álvaro Herrera <alvherre@alvh.no-ip.org>
Sent: Friday, February 26, 2021 2:49 AM
To: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>; Jamison, Kirk/ジャミソン カーク <k.jamison@fujitsu.com>
Cc: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>; 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>; pgsql-hackers@lists.postgresql.org
Subject: Re: libpq debug log

I tweaked this code a little bit more. I didn't like that libpq-trace.h was exposing all the internal details of the API to the world; and I liked it even less that libpq-int.h had to include the file, exposing everything everywhere. I added some forward struct declarations in libpq-int.h to avoid including libpq-trace.h in it; and I also moved a few of the definitions from libpq-trace.h to the .c file. We didn't really need them to be exposed outside the .c file. In the makefile, libpq-trace.h was being installed in the wrong place -- I changed so that it goes to includedir_internal, like libpq-int.h.

The function pqLogFormatTimestamp was returning a pointer to where it printed things. But that was pointless, because the buffer is caller supplied, so the caller knows full well where the output is. There's no gain in functionality, so I made it return void.

I added a description of the output format to the documentation.

I renamed a lot of functions, so that it looks more like a consistent API. pqTraceMaybeBreakLine() says it returns bool, but it was using "return 1/0" rather than actual bools. That's poor style.

The only thing we're missing here is some coverage. Right now, nothing in the source tree calls this at all, so if we break it, we'll never know. I think we should introduce src/test/modules/test_pqtrace or something like that.

v22 attached.

#125Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Álvaro Herrera (#123)
Re: libpq debug log

At Thu, 25 Feb 2021 21:04:00 -0300, Álvaro Herrera <alvherre@alvh.no-ip.org> wrote in

This assert I added?

+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	Assert(conn->be_msg->length > 0);

It breaks. Maybe there's a good reason, maybe there ain't, but in the
meantime I just removed it.

The assertion contradicts the condition just below.

| Assert(conn->be_msg->length > 0);
|
| conn->be_msg->length -= size;
| if (conn->be_msg->length <= 0)

It seems to me that the assertion was inteded to be placed after
subtracting size, and the operator > is inteded to be >=.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#126Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Kyotaro Horiguchi (#125)
Re: libpq debug log

At Fri, 26 Feb 2021 13:35:26 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

At Thu, 25 Feb 2021 21:04:00 -0300, Álvaro Herrera <alvherre@alvh.no-ip.org> wrote in

This assert I added?

+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	Assert(conn->be_msg->length > 0);

It breaks. Maybe there's a good reason, maybe there ain't, but in the
meantime I just removed it.

The assertion contradicts the condition just below.

Sorry, I was out of it. Please forget that.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#127Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: iwata.aya@fujitsu.com (#124)
1 attachment(s)
Re: libpq debug log

(we had better avoid top-posting:p)

At Fri, 26 Feb 2021 02:52:49 +0000, "iwata.aya@fujitsu.com" <iwata.aya@fujitsu.com> wrote in

Alvaro san,

Thank you very much for your updating and organizing this patch.

It appears that something is still wrong. I applied lipq pipeline v27 from [1] and ran src/test/modules/test_libpq/pipeline singlerow, after patching it to do PQtrace() after PQconn(). Below is the output I get from that. The noteworthy point is that "ParseComplete" messages appear multiple times for some reason ... but that's quite odd, because if I look at the network traffic with Wireshark I certainly do not see the ParseComplete message being sent three times.

I will search this cause. Please wait a minuets.
I thought I could avoid multiple such outputs by using conn->LogCursor…

I found a bug of the patch.

- Tweaked psql to enable tracing. (attached)

- Connect to a server using SSL.

- Restart the server.
<many ":::Invalid Protocol" are emitted>

- Reconnect to the server by just entering ';<ret>' on the psql
command line. I saw the following log lines.

2021-02-26 14:29:40.434749 > Query 6 ";"
2021-02-26 14:29:40.435948 < ErrorResponse 116 S "FATAL" V "FATAL" C "57P01" M "terminating connection due to administrator command" F "postgres.c" L "3129" R "ProcessInterrupts" \x00
2021-02-26 14:29:40.438298 > SSLRequest 8 '\x04\xffffffd2\x16/'
2021-02-26 14:29:40.443155 < ParameterStatus 2021-02-26 14:29:40.468186 > StartupMessage 84 '\x00\x03\x00\x00user\x00horiguti\x00database\x00postgres\x00application_name\x00psql\x00client_encoding\x00UTF8\x00\x00'
< :::Invalid Protocol
< :::Invalid Protocol

1. It seems to have desynced.
2. ":::Invalid Protocol" looks somewhat odd?
3. ":::Invalid Protocol" lines missing a timestamp.

- type "select 1;<ret>"
<several ":::Invalid Protocols" then psql crashes>

I haven't looked further, though.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachments:

psqltracing.diff.txttext/plain; charset=us-asciiDownload
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..1364e1aa13 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -25,7 +25,7 @@
 #include "input.h"
 #include "mainloop.h"
 #include "settings.h"
-
+#include "libpq-trace.h"
 /*
  * Global psql options
  */
@@ -300,6 +300,12 @@ main(int argc, char *argv[])
 		exit(EXIT_BADCONN);
 	}
 
+	{
+		FILE *fp = fopen("hoge.trc", "a");
+		Assert (fp);
+		PQtrace(pset.db, fp);
+	}
+	
 	psql_setup_cancel_handler();
 
 	PQsetNoticeProcessor(pset.db, NoticeProcessor, NULL);
#128Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Kyotaro Horiguchi (#127)
Re: libpq debug log

(Is what I'm drinking now decafe?)

At Fri, 26 Feb 2021 14:33:26 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

- Tweaked psql to enable tracing. (attached)

- Connect to a server using SSL.

- Restart the server.

-> <many ":::Invalid Protocol" are emitted>

This is a mistake. Just restaring the server doesn't cause this.

- Reconnect to the server by just entering ';<ret>' on the psql
command line. I saw the following log lines.

2021-02-26 14:29:40.434749 > Query 6 ";"
2021-02-26 14:29:40.435948 < ErrorResponse 116 S "FATAL" V "FATAL" C "57P01" M "terminating connection due to administrator command" F "postgres.c" L "3129" R "ProcessInterrupts" \x00
2021-02-26 14:29:40.438298 > SSLRequest 8 '\x04\xffffffd2\x16/'
2021-02-26 14:29:40.443155 < ParameterStatus 2021-02-26 14:29:40.468186 > StartupMessage 84 '\x00\x03\x00\x00user\x00horiguti\x00database\x00postgres\x00application_name\x00psql\x00client_encoding\x00UTF8\x00\x00'
< :::Invalid Protocol
< :::Invalid Protocol

1. It seems to have desynced.
2. ":::Invalid Protocol" looks somewhat odd?
3. ":::Invalid Protocol" lines missing a timestamp.

- type "select 1;<ret>"
<several ":::Invalid Protocols" then psql crashes>

I haven't looked further, though.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#129tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Álvaro Herrera (#122)
RE: libpq debug log

From: Álvaro Herrera <alvherre@alvh.no-ip.org>

It appears that something is still wrong. I applied lipq pipeline v27 from [1] and ran src/test/modules/test_libpq/pipeline singlerow, after patching it to do PQtrace() after PQconn(). Below is the output I get from that. The noteworthy point is that "ParseComplete" messages appear multiple times for some reason ... but that's quite odd, because if I look at the network traffic with Wireshark I certainly do not see the ParseComplete message being sent three times.

< CommandComplete 13 "SELECT 3"
< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4

Hmm, that's mysterious. I'm not sure if this explains why, but some places such as pqTraceOutputBeByte1() and pqTraceOutputString() appears to forget to update LogCursor.

In addition, in pqTraceOutputBeInt(),

+ bool logfinish = 0;

0 should be false instead.

+ logfinish = pqTraceMaybeBreakLine(0, conn);

"length" should be passed instead of 0, shouldn't it?

Regards
Takayuki Tsunakawa

#130Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#129)
Re: libpq debug log

At Fri, 26 Feb 2021 06:21:15 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

From: Álvaro Herrera <alvherre@alvh.no-ip.org>

It appears that something is still wrong. I applied lipq pipeline v27 from [1] and ran src/test/modules/test_libpq/pipeline singlerow, after patching it to do PQtrace() after PQconn(). Below is the output I get from that. The noteworthy point is that "ParseComplete" messages appear multiple times for some reason ... but that's quite odd, because if I look at the network traffic with Wireshark I certainly do not see the ParseComplete message being sent three times.

< CommandComplete 13 "SELECT 3"
< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4

Hmm, that's mysterious. I'm not sure if this explains why, but some places such as pqTraceOutputBeByte1() and pqTraceOutputString() appears to forget to update LogCursor.

In addition, in pqTraceOutputBeInt(),

+ bool logfinish = 0;

0 should be false instead.

+ logfinish = pqTraceMaybeBreakLine(0, conn);

"length" should be passed instead of 0, shouldn't it?

The reason is be_msg->length is set with the value already subtracted
length. Perhaps the following change allows the Assert() to
resurrect.

 +		case LOG_LENGTH:
 +			fprintf(conn->Pfdebug, "%d", v);
-+			conn->be_msg->length = v - length;
++			conn->be_msg->length = v;
 +			/* Next, log the message contents */
 +			conn->be_msg->state = LOG_CONTENTS;
-+			logfinish = pqTraceMaybeBreakLine(0, conn);
++			logfinish = pqTraceMaybeBreakLine(length, conn);

However, This doesn't seem cause the phenomenon.

+	if (!logfinish)
+		conn->be_msg->logCursor = conn->inCursor - conn->inStart;

This inhibits logCursor being updated. What is worse, I find that
logCursor movement is quite dubious.

pqTraceOutputBeByte1 donsn't move logCursor for LOG_FIRST_BYTE
pqTraceOutputBeInt doesn't move logCursor if the messages *does* end.

So it repeats the same message if the previous message consists of
<type, length=4>.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#131Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Kyotaro Horiguchi (#130)
Re: libpq debug log

At Fri, 26 Feb 2021 16:12:39 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

This inhibits logCursor being updated. What is worse, I find that
logCursor movement is quite dubious.

Using (inCursor - inStart) as logCursor doesn't work correctly if
tracing state desyncs. Once desync happens inStart can be moved at
the timing that the tracing code doesn't expect. This requires (as I
mentioned upthread) pqReadData to actively reset logCursor, though.

logCursor should move when bytes are fed to the tracing functoins even
when theyare the last bytes of a message.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#132Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Kyotaro Horiguchi (#131)
Re: libpq debug log

At Fri, 26 Feb 2021 17:30:32 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

At Fri, 26 Feb 2021 16:12:39 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

This inhibits logCursor being updated. What is worse, I find that
logCursor movement is quite dubious.

Using (inCursor - inStart) as logCursor doesn't work correctly if
tracing state desyncs. Once desync happens inStart can be moved at
the timing that the tracing code doesn't expect.

-                                                   This requires (as I
- mentioned upthread) pqReadData to actively reset logCursor, though.
+ So pgReadData needs to avtively reset logCursor.  If logCursor is
+ actively reset, we no longer need to use (inCursor - inStart) as
+ logCursor and it is enough that logCursor follows inCursor.

logCursor should move when bytes are fed to the tracing functoins even
when theyare the last bytes of a message.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#133tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#131)
RE: libpq debug log

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

Using (inCursor - inStart) as logCursor doesn't work correctly if tracing state
desyncs. Once desync happens inStart can be moved at the timing that the
tracing code doesn't expect. This requires (as I mentioned upthread)
pqReadData to actively reset logCursor, though.

logCursor should move when bytes are fed to the tracing functoins even when
theyare the last bytes of a message.

Hmm, sharp and nice insight. I'd like Iwata-san and Kirk to consider this, including Horiguchi-san's previous mails.

Regards
Takayuki Tsunakawa

#134Tom Lane
tgl@sss.pgh.pa.us
In reply to: tsunakawa.takay@fujitsu.com (#133)
Re: libpq debug log

"tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> writes:

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

Using (inCursor - inStart) as logCursor doesn't work correctly if tracing state
desyncs. Once desync happens inStart can be moved at the timing that the
tracing code doesn't expect. This requires (as I mentioned upthread)
pqReadData to actively reset logCursor, though.

logCursor should move when bytes are fed to the tracing functoins even when
theyare the last bytes of a message.

Hmm, sharp and nice insight. I'd like Iwata-san and Kirk to consider this, including Horiguchi-san's previous mails.

I took a quick look through the v22 patch, and TBH I don't like much of
anything at all about the proposed architecture. It's retained most
of the flavor of the way it was done before, which was a hangover from
the old process-on-the-fly scheme.

I think the right way to do this, now that we've killed off v2 protocol
support, is to rely on the fact that libpq fully buffers both incoming
and outgoing messages. We should nuke every last bit of the existing
code that intersperses tracing logic with normal message decoding and
construction, and instead have two tracing entry points:

(1) Log a received message. This is called as soon as we know (from
the length word) that we have the whole message available, and it's
just passed a pointer into the input buffer. It should examine the
input message for itself and print the details.

(2) Log a message-to-be-sent. Again, call it as soon as we've constructed
a complete message in the output buffer, and let it examine and print
that message for itself.

Both of these pieces of logic could be written directly from the protocol
specification, without any interaction with the main libpq code paths,
which would be a Good Thing IMO. The current intertwined approach is
somewhere between useless and counterproductive if you're in need of
tracing down a libpq bug. (In other words, I'm suggesting that the
apparent need for duplicate logic would be good not bad, and indeed that
it'd be best to write the tracing logic without consulting the existing
libpq code at all.)

This would, I think, also eliminate the need for extra storage to hold
info about bits of the message, which IMO is a pretty significant
downside of the proposed design. The printing logic would just print
directly to Pfdebug; it wouldn't have to accumulate anything, and so
it wouldn't have the out-of-memory failure modes that this patch has.
You could also get rid of messy stuff like pqTraceForciblyCloseBeMsg.

Lastly, this'd reduce the overhead the tracing functionality imposes
on normal usage to about nil. Admittedly, all those "if (Pfdebug)"
tests probably don't cost that much execution-wise, but they cost
something. Making only one such test per sent or received message
has to be better.

regards, tom lane

#135tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Tom Lane (#134)
RE: libpq debug log

Tom-san, Alvaro-san,

From: Tom Lane <tgl@sss.pgh.pa.us>

I took a quick look through the v22 patch, and TBH I don't like much of anything
at all about the proposed architecture. It's retained most of the flavor of the
way it was done before, which was a hangover from the old process-on-the-fly
scheme.

Yes, the patch just follows the current style of interspersing. (But some places are removed so it's a bit cleaner.)

Anyway, I think your suggestion should be better, resulting in much cleaner separation of message handling and logging.

I think the right way to do this, now that we've killed off v2 protocol support, is to
rely on the fact that libpq fully buffers both incoming and outgoing messages.
We should nuke every last bit of the existing code that intersperses tracing logic
with normal message decoding and construction, and instead have two tracing
entry points:

FWIW, I was surprised and glad when I saw the commit message to remove the v2 protocol.

(1) Log a received message. This is called as soon as we know (from the
length word) that we have the whole message available, and it's just passed a
pointer into the input buffer. It should examine the input message for itself
and print the details.

(2) Log a message-to-be-sent. Again, call it as soon as we've constructed a
complete message in the output buffer, and let it examine and print that
message for itself.

I understood that the former is pqParseInput3() and the latter is pqPutMsgEnd(). They call the logging function wne conn->pfdebug is not NULL. Its signature is like this (that will be defined in libpq-trace.c):

void pqLogMessage(const char *message, bool toServer);

The message argument points to the message type byte. toServer is true for messages sent to the server. The signature could alternatively be one of the following, but this is a matter of taste:

void pqLogMessage(char message_type, const char *message, bool toServer);
void pqLogMessage(char message_type, int length, const char *message, bool toServer);

This function simply outputs the timestamp, message direction, message type, length, and message contents. The output processing of message contents branches on a message type with a switch-case statement. Perhaps the output processing of each message should be separated into its own function like pqLogMessageD() for 'D' message so that the indentation won't get deep in the main logging function.

Both of these pieces of logic could be written directly from the protocol
specification, without any interaction with the main libpq code paths, which
would be a Good Thing IMO. The current intertwined approach is somewhere
between useless and counterproductive if you're in need of tracing down a
libpq bug. (In other words, I'm suggesting that the apparent need for
duplicate logic would be good not bad, and indeed that it'd be best to write the
tracing logic without consulting the existing libpq code at all.)

Yes, the only thing I'm concerned about is to have two places that have to be aware of the message format -- message encoding/decoding and logging. But this cleaner separation would pay it.

Alvaro-san, what do you think? Anyway, we'll soon post the updated patch that fixes the repeated ParseComplete issue based on the current patch. After that, we'll send a patch based on Tom-san's idea.

Regards
Takayuki Tsunakawa

#136tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#135)
RE: libpq debug log

From: tsunakawa.takay@fujitsu.com <tsunakawa.takay@fujitsu.com>

I understood that the former is pqParseInput3() and the latter is
pqPutMsgEnd(). They call the logging function wne conn->pfdebug is not
NULL. Its signature is like this (that will be defined in libpq-trace.c):

void pqLogMessage(const char *message, bool toServer);

Oops, the logging facility needs the destination (conn->pfdebug). So, it will be:

void pqLogMessage(PGconn *conn, bool toServer);

The function should know where to extract the message from.

Regards
Takayuki Tsunakawa

#137Tom Lane
tgl@sss.pgh.pa.us
In reply to: tsunakawa.takay@fujitsu.com (#136)
Re: libpq debug log

"tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> writes:

Oops, the logging facility needs the destination (conn->pfdebug). So, it will be:

void pqLogMessage(PGconn *conn, bool toServer);

Right, and it'd need the debug flags too, so +1 for just passing the
PGconn instead of handing those over piecemeal.

But I think passing the message start address explicitly might be
better than having it understand the buffering behavior in enough
detail to know where to find the message. Part of the point here
(IMO) is to decouple the tracing logic from the core libpq logic,
in hopes of not having common-mode bugs.

regards, tom lane

#138tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Tom Lane (#137)
RE: libpq debug log

From: Tom Lane <tgl@sss.pgh.pa.us>

But I think passing the message start address explicitly might be better than
having it understand the buffering behavior in enough detail to know where to
find the message. Part of the point here
(IMO) is to decouple the tracing logic from the core libpq logic, in hopes of not
having common-mode bugs.

Ouch, you're perfectly right. Then let's make the signature:

void pqLogMessage(PGconn *conn, const char *message, bool toServer);

Thank you!

Regards
Takayuki Tsunakawa

#139k.jamison@fujitsu.com
k.jamison@fujitsu.com
In reply to: Kyotaro Horiguchi (#132)
1 attachment(s)
RE: libpq debug log

Hi Alvaro-san and Horiguchi-san,
CC) Iwata-san, Tsunakawa-san

Attached is the V23 of the libpq trace patch.

(1)
From: Álvaro Herrera <alvherre@alvh.no-ip.org>

It appears that something is still wrong. I applied lipq pipeline v27 from [1]
and ran src/test/modules/test_libpq/pipeline singlerow, after patching it to
do PQtrace() after PQconn(). Below is the output I get from that. The
noteworthy point is that "ParseComplete" messages appear multiple times
for some reason ... but that's quite odd, because if I look at the network traffic
with Wireshark I certainly do not see the ParseComplete message being sent
three times.

I applied Alvaro's libpq pipeline patch (v33) [1]/messages/by-id/20210304214017.GA22915@alvherre.pgsql on top of this libpq trace patch,
then traced the src/test/modules/libpq_pipeline/libpq_pipeline.c test_singlerowmode.
It's still the same trace output even after applying the fix recommendations
of Horiguchi-san.

Parse 38 "" "SELECT generate_series(42, $1)" #0
Bind 20 "" "" #0 #1 2 '44' #1 #0
Describe 6 P ""
Execute 9 "" 0
Parse 38 "" "SELECT generate_series(42, $1)" #0
Bind 20 "" "" #0 #1 2 '45' #1 #0
Describe 6 P ""
Execute 9 "" 0
Parse 38 "" "SELECT generate_series(42, $1)" #0
Bind 20 "" "" #0 #1 2 '46' #1 #0
Describe 6 P ""
Execute 9 "" 0
Sync 4

< ParseComplete 4
< BindComplete 4
< RowDescription 40 #1 "generate_series" 0 #0 23 #4 -1 #0
< DataRow 12 #1 2 '42'
< DataRow 12 #1 2 '43'
< DataRow 12 #1 2 '44'
< CommandComplete 13 "SELECT 3"
< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4
< RowDescription 40 #1 "generate_series" 0 #0 23 #4 -1 #0
< DataRow 12 #1 2 '42'
< DataRow 12 #1 2 '43'
< DataRow 12 #1 2 '44'
< DataRow 12 #1 2 '45'
< CommandComplete 13 "SELECT 4"
< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4
< RowDescription 40 #1 "generate_series" 0 #0 23 #4 -1 #0
< DataRow 12 #1 2 '42'
< DataRow 12 #1 2 '43'
< DataRow 12 #1 2 '44'
< DataRow 12 #1 2 '45'
< DataRow 12 #1 2 '46'
< CommandComplete 13 "SELECT 5"
< ReadyForQuery 5 I

ParseComplete still appears 3 times, but I wonder if that is expected only for pipeline's singlerowmode test.
From my observation, when I traced the whole regression test output
by putting PQtrace() in fe-connect.c: connectDBComplete(),
ParseComplete appears only once or twice, etc. for other commands.
In other words, ISTM, it's a case-to-case basis:
Some examples from the whole regression trace output is below:
a.

Describe 6 P ""
Execute 9 "" 0
Sync 4

< ParseComplete 4
< BindComplete 4
b.
< ErrorResponse 217 S "ERROR" V "ERROR" C "42883" M "function no_such_function(integer) does not exist" H "No function matches the given name and argument types. You might need to add explicit type casts." P "8" F "parse_func.c" L "629" R "ParseFuncOrColumn" \x00
< ReadyForQuery 5 I
< ParseComplete 4
< ParseComplete 4
c.

Bind 31 "" "my_insert" #0 #1 4 '9652' #1 #0
Describe 6 P ""
Execute 9 "" 0

< ParseComplete 4
< ParseComplete 4
< ParseComplete 4
< BindComplete 4
< NoData 4
d.
< CommandComplete 15 "INSERT 0 2"
< ReadyForQuery 5 T

Parse 89 "i" "insert into test (name, amount, letter) select name, amount+$1, letter from test" #0
Sync 4

< ParseComplete 4
< ReadyForQuery 5 T

As you can see from the examples above, ParseComplete appears in multiples/once depending on the test.
As for libpq_pipeline's test_singlerowmode, it appears 3x for the CommandComplete SELECT.
I am wondering now whether or not it's a bug. Maybe it's not.
Thoughts?

How I traced the whole regression test: fe-connect.c: connectDBComplete()

@@ -2114,6 +2137,7 @@ connectDBComplete(PGconn *conn)
int timeout = 0;
int last_whichhost = -2; /* certainly different from whichhost */
struct addrinfo *last_addr_cur = NULL;
+ FILE *trace_file;

 	if (conn == NULL || conn->status == CONNECTION_BAD)
 		return 0;
@@ -2171,6 +2195,9 @@ connectDBComplete(PGconn *conn)
 		switch (flag)
 		{
 			case PGRES_POLLING_OK:
+				trace_file = fopen("/home/postgres/tracelog/regression_tracepipeline_a.out","a");
+				PQtrace(conn, trace_file);
+				PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);

(2)

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

This inhibits logCursor being updated. What is worse, I find that
logCursor movement is quite dubious.

Using (inCursor - inStart) as logCursor doesn't work correctly if
tracing state desyncs. Once desync happens inStart can be moved at
the timing that the tracing code doesn't expect.

-                                                   This requires (as I
- mentioned upthread) pqReadData to actively reset logCursor, though.
+ So pgReadData needs to avtively reset logCursor.  If logCursor is
+ actively reset, we no longer need to use (inCursor - inStart) as
+ logCursor and it is enough that logCursor follows inCursor.

logCursor should move when bytes are fed to the tracing functoins even
when theyare the last bytes of a message.

Following the advice of Horiguchi-san above, of updating logCursor through pqReadData,
I have updated pqReadData and pqCheckInBufferSpace.
In pqTraceOutputBeByte1 and pqTraceOutputBeInt.
logCursor now uses the inCursor value because pqReadData actively resets the logCursor.

In addition, I also fixed the following:

(3)
In pqTraceOutputBeByte1, I added the missing update of logCursor for the LOG_FIRST_BYTE:

+        case LOG_FIRST_BYTE:
+            {
+                ...
+                conn->be_msg->logCursor = conn->inCursor;
+                break;
+            }

(4)
In pqTraceOutputBeInt():

I changed 0 to false:
-    bool        logfinish = 0;
+    bool        logfinish = false;

I followed this advice and applied to changes for LOG_LENGTH:

+        case LOG_LENGTH:
+            fprintf(conn->Pfdebug, "%d", v);
-+            conn->be_msg->length = v - length;
++            conn->be_msg->length = v;
+            /* Next, log the message contents */
+            conn->be_msg->state = LOG_CONTENTS;
-+            logfinish = pqTraceMaybeBreakLine(0, conn);
++            logfinish = pqTraceMaybeBreakLine(length, conn);

(5)
Because of the changes in pqReadData, I exposed again the
following and moved from libpq-trace.c to libq-trace.h:
+typedef enum PGLogState
+typedef struct pqBackendMessage

(6)
As reported abovethread, the Assert in pqTraceMaybeBreakLine() still breaks. So I removed it

psql: libpq-trace.c:320: pqTraceMaybeBreakLine: Assertion `conn->be_msg->length > 0' failed.

(7) Not included in this patch is a fix for the bug I found from the whole regression test output.
There are many instances of Invalid Protocol, but only for FunctionCallResponse

< FunctionCallResponse 8 0
'< :::Invalid Protocol
'

See fe-protocol3.c: pqFunctionCall3().
Perhaps it has something to do when the actual_result_len == 0, and it's not integer (pqGetnchar).
And somewhere among pqTraceOutputBeInt(), pqTraceMaybeBreakLine(), etc.
probably breaks something when message is V, 8, 0 \0.
(Referring to (6) above, maybe the reason why Assert(conn->be_msg->length > 0) fails
in pqTraceMaybeBreakLine() is because the length may possibly be 0.)

Thoughts?

fe-protocol3.c: pqFunctionCall3():
case 'V': /* function result */
if (pqGetInt(actual_result_len, 4, conn))
continue;
if (*actual_result_len != -1)
{
if (result_is_int)
{
if (pqGetInt(result_buf, *actual_result_len, conn))
continue;
}
else
{
if (pqGetnchar((char *) result_buf,
*actual_result_len,
conn))
continue;
}
}

(8) Regarding the exchange of discussions between Tom Lane and Tsunakawa-san
about the improved tracing design, Iwata-san is now working on it and
will submit the updated patch soon.

Regards,
Kirk Jamison

[1]: /messages/by-id/20210304214017.GA22915@alvherre.pgsql

Attachments:

v23-0001-Improve-libpq-tracing-capabilities.patchapplication/octet-stream; name=v23-0001-Improve-libpq-tracing-capabilities.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 0553279..485ad8c 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5934,12 +5934,28 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
+      Only available when protocol version 3 or higher is used.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message type, message length, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      For numerical quantities, 16-bit values are preceded with <literal>#</literal>
+      and 32-bit values are printed without a prefix.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -5954,6 +5970,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..2311399 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -113,6 +114,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -126,6 +128,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir_internal)/libpq-trace.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index f83af03..1d0e689 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4097,6 +4097,10 @@ freePGconn(PGconn *conn)
 	if (conn->connip)
 		free(conn->connip);
 	/* Note that conn->Pfdebug is not ours to close or free */
+	if (conn->be_msg)
+		free(conn->be_msg);
+	if (conn->fe_msg)
+		free(conn->fe_msg);
 	if (conn->last_query)
 		free(conn->last_query);
 	if (conn->write_err_msg)
@@ -6835,27 +6839,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 9a03804..5009e98 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..9ceff73 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -68,7 +69,6 @@ PQlibVersion(void)
 	return PG_VERSION_NUM;
 }
 
-
 /*
  * pqGetc: get 1 character from the connection
  *
@@ -85,12 +85,11 @@ pqGetc(char *result, PGconn *conn)
 	*result = conn->inBuffer[conn->inCursor++];
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
+		pqTraceOutputBeByte1(conn, *result);
 
 	return 0;
 }
 
-
 /*
  * pqPutc: write 1 char to the current message
  */
@@ -101,12 +100,11 @@ pqPutc(char c, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
+		pqTraceStoreFeMsg(conn, LOG_BYTE1, 1);
 
 	return 0;
 }
 
-
 /*
  * pqGets[_append]:
  * get a null-terminated string from the connection,
@@ -139,8 +137,7 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 	conn->inCursor = ++inCursor;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
+		pqTraceOutputString(conn, buf->data, buf->len + 1, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -157,18 +154,19 @@ pqGets_append(PQExpBuffer buf, PGconn *conn)
 	return pqGets_internal(buf, conn, false);
 }
 
-
 /*
  * pqPuts: write a null-terminated string to the current message
  */
 int
 pqPuts(const char *s, PGconn *conn)
 {
-	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
+	int			length = strlen(s) + 1;
+
+	if (pqPutMsgBytes(s, length, conn))
 		return EOF;
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
+		pqTraceStoreFeMsg(conn, LOG_STRING, length);
 
 	return 0;
 }
@@ -189,11 +187,7 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 	conn->inCursor += len;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceOutputNchar(conn, s, len, MSGDIR_FROM_BACKEND);
 
 	return 0;
 }
@@ -213,12 +207,8 @@ pqSkipnchar(size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
+		pqTraceOutputNchar(conn, conn->inBuffer + conn->inCursor, len,
+						   MSGDIR_FROM_BACKEND);
 	conn->inCursor += len;
 
 	return 0;
@@ -235,11 +225,7 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 		return EOF;
 
 	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
+		pqTraceStoreFeMsg(conn, LOG_NCHAR, len);
 
 	return 0;
 }
@@ -279,7 +265,7 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
+		pqTraceOutputBeInt(conn, *result, (unsigned int) bytes);
 
 	return 0;
 }
@@ -294,15 +280,18 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 {
 	uint16		tmp2;
 	uint32		tmp4;
+	PGLogMsgDataType type;
 
 	switch (bytes)
 	{
 		case 2:
+			type = LOG_INT16;
 			tmp2 = pg_hton16((uint16) value);
 			if (pqPutMsgBytes((const char *) &tmp2, 2, conn))
 				return EOF;
 			break;
 		case 4:
+			type = LOG_INT32;
 			tmp4 = pg_hton32((uint32) value);
 			if (pqPutMsgBytes((const char *) &tmp4, 4, conn))
 				return EOF;
@@ -315,7 +304,7 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 	}
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
+		pqTraceStoreFeMsg(conn, type, (unsigned int) bytes);
 
 	return 0;
 }
@@ -417,6 +406,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 					conn->inEnd - conn->inStart);
 			conn->inEnd -= conn->inStart;
 			conn->inCursor -= conn->inStart;
+			if (conn->Pfdebug)
+				conn->be_msg->logCursor -= conn->inStart;
 			conn->inStart = 0;
 		}
 	}
@@ -424,6 +415,8 @@ pqCheckInBufferSpace(size_t bytes_needed, PGconn *conn)
 	{
 		/* buffer is logically empty, reset it */
 		conn->inStart = conn->inCursor = conn->inEnd = 0;
+		if (conn->Pfdebug)
+			conn->be_msg->logCursor = 0;
 	}
 
 	/* Recheck whether we have enough space */
@@ -526,8 +519,7 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
 	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
+		pqTraceSetFeMsgType(conn, msg_type);
 
 	return 0;
 }
@@ -563,15 +555,14 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
 		uint32		msgLen = conn->outMsgEnd - conn->outMsgStart;
 
+		if (conn->Pfdebug)
+			pqTraceOutputFeMsg(conn, msgLen);
+
 		msgLen = pg_hton32(msgLen);
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
@@ -624,6 +615,8 @@ pqReadData(PGconn *conn)
 					conn->inEnd - conn->inStart);
 			conn->inEnd -= conn->inStart;
 			conn->inCursor -= conn->inStart;
+			if (conn->Pfdebug)
+				conn->be_msg->logCursor -= conn->inStart;
 			conn->inStart = 0;
 		}
 	}
@@ -631,6 +624,8 @@ pqReadData(PGconn *conn)
 	{
 		/* buffer is logically empty, reset it */
 		conn->inStart = conn->inCursor = conn->inEnd = 0;
+		if (conn->Pfdebug)
+			conn->be_msg->logCursor = 0;
 	}
 
 	/*
@@ -1002,11 +997,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 2ca8c05..9893b09 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -28,6 +28,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "port/pg_bswap.h"
 
@@ -283,6 +284,9 @@ pqParseInput3(PGconn *conn)
 						 * the data till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForciblyCloseBeMsg(msgLength, conn);
 					}
 					else if (conn->result == NULL ||
 							 conn->queryclass == PGQUERY_DESCRIBE)
@@ -357,6 +361,9 @@ pqParseInput3(PGconn *conn)
 						 * tuples till we get to the end of the query.
 						 */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForciblyCloseBeMsg(msgLength, conn);
 					}
 					else
 					{
@@ -366,6 +373,9 @@ pqParseInput3(PGconn *conn)
 						pqSaveErrorResult(conn);
 						/* Discard the unexpected message */
 						conn->inCursor += msgLength;
+						/* Terminate a half-finished logging message */
+						if (conn->Pfdebug)
+							pqTraceForciblyCloseBeMsg(msgLength, conn);
 					}
 					break;
 				case 'G':		/* Start Copy In */
@@ -393,6 +403,9 @@ pqParseInput3(PGconn *conn)
 					 * early.
 					 */
 					conn->inCursor += msgLength;
+					/* Terminate a half-finished logging message */
+					if (conn->Pfdebug)
+						pqTraceForciblyCloseBeMsg(msgLength, conn);
 					break;
 				case 'c':		/* Copy Done */
 
@@ -454,6 +467,9 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
 	/* flush input data since we're giving up on processing it */
 	pqDropConnection(conn, true);
 	conn->status = CONNECTION_BAD;	/* No more connection to backend */
+	/* Terminate a half-finished logging message */
+	if (conn->Pfdebug)
+		pqTraceForciblyCloseBeMsg(msgLength, conn);
 }
 
 /*
@@ -1620,6 +1636,9 @@ getCopyDataMessage(PGconn *conn)
 					return 0;
 				break;
 			case 'd':			/* Copy Data, pass it back to caller */
+				/* Terminate a half-finished logging message */
+				if (conn->Pfdebug)
+					pqTraceForciblyCloseBeMsg(msgLength, conn);
 				return msgLength;
 			case 'c':
 
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index fa9b62a..59ea464 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -362,7 +362,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 8d51e6e..f4033ce 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -321,6 +321,10 @@ typedef struct pg_conn_host
 								 * found in password file. */
 } pg_conn_host;
 
+/* forward declarations */
+struct pqBackendMessage;
+struct pqFrontendMessage;
+
 /*
  * PGconn stores all the state data associated with a single connection
  * to a backend.
@@ -378,6 +382,11 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
+
+	/* pending protocol trace messages */
+	struct pqBackendMessage *be_msg;
+	struct pqFrontendMessage *fe_msg;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..1fb4a7d
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,669 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-trace.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+/*
+ * Frontend messages are accumulated as individual fields that each is a
+ * pointer into conn->outBuffer.
+ */
+typedef struct pqFrontendMessageField
+{
+	PGLogMsgDataType type;
+	int			offset;
+	int			length;
+} pqFrontendMessageField;
+
+typedef struct pqFrontendMessage
+{
+	char		msg_type;
+	int			num_fields;		/* array used size */
+	int			max_fields;		/* array allocated size */
+	pqFrontendMessageField fields[FLEXIBLE_ARRAY_MEMBER];
+} pqFrontendMessage;
+
+#define DEF_FE_MSGFIELDS 256	/* initial fields allocation quantum */
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+
+static bool pqTraceInit(PGconn *conn);
+static bool pqTraceMaybeBreakLine(int size, PGconn *conn);
+static void pqTraceOutputBinary(PGconn *conn, const char *v, int length,
+								PGCommSource commsource);
+static const char *pqGetProtocolMsgType(unsigned char c,
+										PGCommSource commsource);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceOutputFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+	if (pqTraceInit(conn))
+	{
+		setvbuf(debug_port, NULL, _IOLBF, 0);
+		conn->Pfdebug = debug_port;
+	}
+	else
+	{
+		fprintf(debug_port, "Failed to initialize trace support: out of memory\n");
+		fflush(debug_port);
+		conn->Pfdebug = NULL;
+	}
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	/*
+	 * Deallocate FE/BE message tracking memory. If the fe_msg allocation is
+	 * of the initial size, reuse it next time.
+	 */
+	if (conn->fe_msg &&
+		conn->fe_msg->max_fields != DEF_FE_MSGFIELDS)
+	{
+		free(conn->fe_msg);
+		conn->fe_msg = NULL;
+	}
+	if (conn->be_msg)
+	{
+		free(conn->be_msg);
+		conn->be_msg = NULL;
+	}
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->be_msg == NULL || conn->fe_msg == NULL || conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Set up state so that we can trace. NB -- this might be called multiple
+ * times in a process; make sure it's idempotent.
+ */
+static bool
+pqTraceInit(PGconn *conn)
+{
+	conn->traceFlags = 0;
+
+	if (conn->be_msg == NULL)
+	{
+		conn->be_msg = malloc(sizeof(pqBackendMessage));
+		if (conn->be_msg == NULL)
+			return false;
+	}
+
+	if (conn->fe_msg == NULL)
+	{
+		conn->fe_msg = malloc(offsetof(pqFrontendMessage, fields) +
+							  DEF_FE_MSGFIELDS * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			free(conn->be_msg);
+			/* NULL out for the case that fe_msg malloc fails */
+			conn->be_msg = NULL;
+			return false;
+		}
+		conn->fe_msg->max_fields = DEF_FE_MSGFIELDS;
+	}
+
+	conn->fe_msg->num_fields = 0;
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->command = '\0';
+	conn->be_msg->logCursor = 0;
+
+	return true;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage";
+}
+
+/* pqTraceResetBeMsg: Initialize backend message */
+static void
+pqTraceResetBeMsg(PGconn *conn)
+{
+	conn->be_msg->state = LOG_FIRST_BYTE;
+	conn->be_msg->length = 0;
+	conn->be_msg->logCursor = 0;
+}
+
+/* pqLogInvalidProtocol: Print that the protocol message is invalid */
+static void
+pqLogInvalidProtocol(PGconn *conn, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug,
+			"%s\t:::Invalid Protocol\n",
+			commsource == MSGDIR_FROM_BACKEND ? "<" : ">");
+	conn->be_msg->state = LOG_FIRST_BYTE;
+}
+
+/*
+ * pqTraceMaybeBreakLine:
+ *		Check whether the backend message is complete. If so, print a line
+ *		break, reset the buffer, and return true.  If there's still more
+ *		in the backend message, print a blank and return false.
+ */
+static bool
+pqTraceMaybeBreakLine(int size, PGconn *conn)
+{
+	conn->be_msg->length -= size;
+	if (conn->be_msg->length <= 0)
+	{
+		fprintf(conn->Pfdebug, "\n");
+		pqTraceResetBeMsg(conn);
+		return true;
+	}
+	else
+	{
+		fprintf(conn->Pfdebug, " ");
+		return false;
+	}
+}
+
+/*
+ * pqTraceForciblyCloseBeMsg
+ * 		Print a newline and reset backend message state.
+ *
+ * To be used when dealing with various errors.
+ */
+void
+pqTraceForciblyCloseBeMsg(int size, PGconn *conn)
+{
+	fprintf(conn->Pfdebug, "\n");
+	pqTraceResetBeMsg(conn);
+}
+
+void
+pqTraceSetFeMsgType(PGconn *conn, char type)
+{
+	conn->fe_msg->msg_type = type;
+}
+
+/*
+ * pqTraceStoreFeMsg
+ *		Keep track of a from-frontend message that was just written to the
+ *		output buffer.
+ *
+ * Frontend messages are constructed piece by piece, and the message length
+ * is determined at the end, but sent to the server first; so for tracing
+ * purposes we store everything in memory and print to the trace file when
+ * the message is complete.
+ */
+void
+pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length)
+{
+	/* realloc if we've exceeded available space */
+	if (conn->fe_msg->num_fields >= conn->fe_msg->max_fields)
+	{
+		if (conn->fe_msg->max_fields > INT_MAX)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: field message overflow\n");
+			PQuntrace(conn);
+		}
+		conn->fe_msg =
+			realloc(conn->fe_msg,
+					offsetof(pqFrontendMessage, fields) +
+					2 * conn->fe_msg->max_fields * sizeof(pqFrontendMessageField));
+		if (conn->fe_msg == NULL)
+		{
+			fprintf(conn->Pfdebug, "abandoning trace: out of memory\n");
+			PQuntrace(conn);
+			return;
+		}
+		conn->fe_msg->max_fields *= 2;
+	}
+
+	conn->fe_msg->fields[conn->fe_msg->num_fields].type = type;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].offset = conn->outMsgEnd - length;
+	conn->fe_msg->fields[conn->fe_msg->num_fields].length = length;
+	conn->fe_msg->num_fields++;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ * pqTraceOutputFeMsg
+ *		Print accumulated frontend message pieces to the trace file.
+ */
+void
+pqTraceOutputFeMsg(PGconn *conn, int msgLen)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	const char *message_type;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	if (conn->fe_msg->msg_type == '\0')
+	{
+		/*
+		 * We delayed printing message type for special messages; they are
+		 * complete now, so print them.
+		 */
+		if (conn->fe_msg->num_fields > 0)
+		{
+			char	   *message_addr;
+			uint32		result32;
+			int			result;
+
+			message_addr = conn->outBuffer + conn->fe_msg->fields[0].offset;
+			memcpy(&result32, message_addr, 4);
+			result = (int) pg_ntoh32(result32);
+
+			if (result == NEGOTIATE_SSL_CODE)
+				message_type = "SSLRequest";
+			else if (result == NEGOTIATE_GSS_CODE)
+				message_type = "GSSRequest";
+			else
+				message_type = "StartupMessage";
+		}
+		else
+			message_type = "UnknownMessage";
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(conn->fe_msg->msg_type, MSGDIR_FROM_FRONTEND);
+
+	fprintf(conn->Pfdebug, "%s\t>\t%s\t%d",
+			timestr,
+			message_type,
+			msgLen);
+
+	for (int i = 0; i < conn->fe_msg->num_fields; i++)
+	{
+		char	   *message_addr;
+		int			length;
+
+		message_addr = conn->outBuffer + conn->fe_msg->fields[i].offset;
+		length = conn->fe_msg->fields[i].length;
+
+		fprintf(conn->Pfdebug, " ");
+
+		switch (conn->fe_msg->fields[i].type)
+		{
+			case LOG_BYTE1:
+				if (isprint(*message_addr))
+					fprintf(conn->Pfdebug, "%c", *message_addr);
+				else
+					fprintf(conn->Pfdebug, "\\x%02x", *message_addr);
+				break;
+
+			case LOG_STRING:
+				pqTraceOutputString(conn, message_addr,
+									length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_NCHAR:
+				pqTraceOutputNchar(conn, message_addr,
+								   length, MSGDIR_FROM_FRONTEND);
+				break;
+
+			case LOG_INT16:
+				{
+					uint16		result16;
+
+					memcpy(&result16, message_addr, length);
+					result16 = pg_ntoh16(result16);
+					fprintf(conn->Pfdebug, "#%d", result16);
+					break;
+				}
+
+			case LOG_INT32:
+				{
+					uint32		result32;
+
+					memcpy(&result32, message_addr, length);
+					result32 = pg_ntoh32(result32);
+					fprintf(conn->Pfdebug, "%d", result32);
+					break;
+				}
+		}
+	}
+	conn->fe_msg->num_fields = 0;
+
+	fprintf(conn->Pfdebug, "\n");
+}
+
+/*
+ * pqTraceOutputBeByte1: output 1 char from-backend message to the log
+ */
+void
+pqTraceOutputBeByte1(PGconn *conn, char v)
+{
+	if (conn->be_msg->logCursor >= conn->inCursor)
+		return;
+	switch (conn->be_msg->state)
+	{
+		case LOG_FIRST_BYTE:
+			{
+				char		timestr[FORMATTED_TS_LEN];
+
+				if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+					pqTraceFormatTimestamp(timestr, sizeof(timestr));
+				else
+					timestr[0] = '\0';
+				fprintf(conn->Pfdebug, "%s\t<\t%s\t", timestr,
+						pqGetProtocolMsgType((unsigned char) v,
+											 MSGDIR_FROM_BACKEND));
+				/* Next, log the message length */
+				conn->be_msg->state = LOG_LENGTH;
+				conn->be_msg->command = v;
+
+				/* in this state, we must initialize logCursor. */
+				conn->be_msg->logCursor = conn->inCursor;
+				break;
+			}
+
+		case LOG_CONTENTS:
+			/*
+			 * Show non-printable data in hex format, including the
+			 * terminating \0 that completes ErrorResponse and NoticeResponse
+			 * messages.
+			 */
+			if (!isprint(v))
+				fprintf(conn->Pfdebug, "\\x%02x", v);
+			else
+				fprintf(conn->Pfdebug, "%c", v);
+
+			if (!pqTraceMaybeBreakLine(1, conn))
+				conn->be_msg->logCursor = conn->inCursor;
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+}
+
+/*
+ * pqTraceOutputBeInt: output a 2- or 4-byte integer from-backend msg to the log
+ */
+void
+pqTraceOutputBeInt(PGconn *conn, int v, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	bool		logfinish = false;
+
+	if (conn->be_msg->logCursor >= conn->inCursor)
+		return;
+
+	switch (conn->be_msg->state)
+	{
+		case LOG_LENGTH:
+			fprintf(conn->Pfdebug, "%d", v);
+			conn->be_msg->length = v;
+			/* Next, log the message contents */
+			conn->be_msg->state = LOG_CONTENTS;
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		case LOG_CONTENTS:
+			fprintf(conn->Pfdebug, "%s%d", prefix, v);
+			logfinish = pqTraceMaybeBreakLine(length, conn);
+			break;
+
+		default:
+			pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+			break;
+	}
+	if (!logfinish)
+		conn->be_msg->logCursor = conn->inCursor;
+}
+
+
+/*
+ * pqTraceOutputString: output a null-terminated string to the log
+ */
+void
+pqTraceOutputString(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	fprintf(conn->Pfdebug, "\"%s\"", v);
+	if (source == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(length, conn);
+}
+
+/*
+ * pqTraceOutputBinary: output a string possibly consisting of
+ * non-printable characters. Hex representation is used for such
+ * chars; others are printed normally.
+ *
+ * XXX this probably doesn't do a great job with multibyte chars, but then we
+ * don't know what is text and what encoding it'd be in.
+ */
+static void
+pqTraceOutputBinary(PGconn *conn, const char *v, int length, PGCommSource source)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	if (source == MSGDIR_FROM_BACKEND && conn->be_msg->state != LOG_CONTENTS)
+	{
+		pqLogInvalidProtocol(conn, MSGDIR_FROM_BACKEND);
+		return;
+	}
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, conn->Pfdebug);
+			fprintf(conn->Pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, conn->Pfdebug);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+void
+pqTraceOutputNchar(PGconn *conn, const char *v, int len, PGCommSource commsource)
+{
+	fprintf(conn->Pfdebug, "\'");
+	pqTraceOutputBinary(conn, v, len, commsource);
+	fprintf(conn->Pfdebug, "\'");
+	if (commsource == MSGDIR_FROM_BACKEND)
+		pqTraceMaybeBreakLine(len, conn);
+}
diff --git a/src/interfaces/libpq/libpq-trace.h b/src/interfaces/libpq/libpq-trace.h
new file mode 100644
index 0000000..4ab2a55
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.h
@@ -0,0 +1,78 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-trace.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq protocol tracing.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-trace.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_TRACE_H
+#define LIBPQ_TRACE_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+/* Messages from backend */
+typedef enum PGLogState
+{
+	LOG_FIRST_BYTE,				/* logging the message type byte */
+	LOG_LENGTH,					/* logging protocol message length */
+	LOG_CONTENTS				/* logging protocol message contents */
+} PGLogState;
+
+/*
+ * Messages received from backend are printed to the trace file as soon as
+ * processed from the server.
+ */
+typedef struct pqBackendMessage
+{
+	PGLogState	state;			/* state of logging message state machine */
+	char		command;		/* protocol message type */
+	int			length;			/* protocol message length */
+	int			logCursor;		/* next byte of logging */
+} pqBackendMessage;
+
+/* Messages from frontend */
+typedef enum
+{
+	LOG_BYTE1,
+	LOG_STRING,
+	LOG_NCHAR,
+	LOG_INT16,
+	LOG_INT32
+} PGLogMsgDataType;
+
+extern void pqTraceOutputBeByte1(PGconn *conn, char v);
+extern void pqTraceOutputBeInt(PGconn *conn, int v, int length);
+extern void pqTraceOutputString(PGconn *conn, const char *v, int length,
+								PGCommSource commsource);
+extern void pqTraceOutputNchar(PGconn *conn, const char *v, int length,
+							   PGCommSource commsource);
+extern void pqTraceSetFeMsgType(PGconn *conn, char type);
+extern void pqTraceStoreFeMsg(PGconn *conn, PGLogMsgDataType type, int length);
+extern void pqTraceOutputFeMsg(PGconn *conn, int msgLen);
+extern void pqTraceForciblyCloseBeMsg(int size, PGconn *conn);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_TRACE_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..98748f2 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
@@ -1538,6 +1539,8 @@ PGFileType
 PGFunction
 PGLZ_HistEntry
 PGLZ_Strategy
+PGLogMsgDataType
+PGLogState
 PGMessageField
 PGModuleMagicFunction
 PGNoticeHooks
@@ -3275,6 +3278,9 @@ pointer
 polymorphic_actuals
 pos_trgm
 post_parse_analyze_hook_type
+pqBackendMessage
+pqFrontendMessage
+pqFrontendMessageField
 pqbool
 pqsigfunc
 printQueryOpt
#140alvherre@alvh.no-ip.org
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#138)
Re: libpq debug log

On 2021-Mar-05, tsunakawa.takay@fujitsu.com wrote:

From: Tom Lane <tgl@sss.pgh.pa.us>

But I think passing the message start address explicitly might be
better than having it understand the buffering behavior in enough
detail to know where to find the message. Part of the point here
(IMO) is to decouple the tracing logic from the core libpq logic, in
hopes of not having common-mode bugs.

Ouch, you're perfectly right. Then let's make the signature:

void pqLogMessage(PGconn *conn, const char *message, bool toServer);

Yeah, looks good! I agree that going this route will result in more
trustworthy trace output.

--
�lvaro Herrera 39�49'30"S 73�17'W

#141iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: alvherre@alvh.no-ip.org (#140)
1 attachment(s)
RE: libpq debug log

Hi all,

Following all reviewer's advice, I have created a new patch.

In this patch, I add only two tracing entry points; I call pqTraceOutputMsg(PGconn conn, int msgCursor, PGCommSource commsource) in pqParseInput3 () and pqPutMsgEnd () to output log.
The argument contains message first byte offset called msgCursor because it is simpler to specify the buffer pointer in the caller.

In pqTraceOutputMsg(), the content common to all protocol messages (first timestamp, < or >, first 1 byte, and length) are output first and after that each protocol message contents is output. I referred to pqParseInput3 () , fe-exec.c and documentation page for that part.

This fix almost eliminates if(conn->Pfdebug) that was built into the code for other features.

Regards,
Aya Iwata

Show quoted text

-----Original Message-----
From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
Sent: Friday, March 5, 2021 10:41 PM
To: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Cc: 'Tom Lane' <tgl@sss.pgh.pa.us>; Iwata, Aya/岩田 彩
<iwata.aya@fujitsu.com>; Jamison, Kirk/ジャミソン カーク
<k.jamison@fujitsu.com>; 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>;
pgsql-hackers@lists.postgresql.org
Subject: Re: libpq debug log

On 2021-Mar-05, tsunakawa.takay@fujitsu.com wrote:

From: Tom Lane <tgl@sss.pgh.pa.us>

But I think passing the message start address explicitly might be
better than having it understand the buffering behavior in enough
detail to know where to find the message. Part of the point here
(IMO) is to decouple the tracing logic from the core libpq logic, in
hopes of not having common-mode bugs.

Ouch, you're perfectly right. Then let's make the signature:

void pqLogMessage(PGconn *conn, const char *message, bool
toServer);

Yeah, looks good! I agree that going this route will result in more trustworthy
trace output.

--
Álvaro Herrera 39°49'30"S 73°17'W

Attachments:

v24-libpq-trace-log.patchapplication/octet-stream; name=v24-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 910e9a8..eb0c588 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5941,12 +5941,28 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
+      Only available when protocol version 3 or higher is used.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message type, message length, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      For numerical quantities, 16-bit values are preceded with <literal>#</literal>
+      and 32-bit values are printed without a prefix.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -5961,6 +5977,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..2311399 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -113,6 +114,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -126,6 +128,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir_internal)/libpq-trace.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 29054ba..c711669 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6810,27 +6810,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 9a03804..5009e98 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..bfa730f 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -84,9 +85,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +98,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +133,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +158,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +176,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +193,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +208,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +245,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +278,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +486,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +520,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +529,10 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	if(conn->Pfdebug)
+		pqTraceOutputMsg(conn, conn->outMsgStart, MSGDIR_FROM_FRONTEND);
+
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +959,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 2ca8c05..07b4b3b 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -28,6 +28,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "port/pg_bswap.h"
 
@@ -421,6 +422,9 @@ pqParseInput3(PGconn *conn)
 		{
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
+
+			if(conn->Pfdebug)
+				pqTraceOutputMsg(conn, conn->inStart - 5 - msgLength, MSGDIR_FROM_BACKEND);
 		}
 		else
 		{
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index fa9b62a..59ea464 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -362,7 +362,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index adf149a..e35ad19 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -376,6 +376,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..ef294fa
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,689 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-trace.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+#define FORMATTED_TS_LEN 128	/* formatted timestamp length */
+
+static const char *pqGetProtocolMsgType(unsigned char c,
+										PGCommSource commsource);
+static void pqTraceOutputByte1(int LogEnd, int *cursor, FILE *pfdebug, char v);
+static int pqTraceOutputInt(char *buf, int LogEnd,int *cursor,
+													FILE *pfdebug, int length);
+static int pqTraceOutputString(char *buf, int LogEnd,int *cursor, FILE *pfdebug);
+static void pqTraceOutputBinary(char *v, int length, FILE *pfdebug);
+static void pqTraceOutputNchar(char *buf, int LogEnd, int *cursor,
+													FILE *pfdebug, int len);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceOutputFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, PGCommSource commsource)
+{
+	if (commsource == MSGDIR_FROM_BACKEND &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	if (commsource == MSGDIR_FROM_FRONTEND &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	return "UnknownMessage:";
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(int LogEnd, int *cursor, FILE *pfdebug, char v)
+{
+	if (cursor == NULL)
+		return;
+	if (*cursor > LogEnd)
+		return;
+
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(v))
+		fprintf(pfdebug, "\\x%02x", v);
+	else
+		fprintf(pfdebug, "%c", v);
+
+	if (*cursor < LogEnd)
+		fprintf(pfdebug, " ");
+	else
+		fprintf(pfdebug, "\n");
+}
+
+/*
+ *   pqTraceOutputInt: output a 2- or 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt(char *buf, int LogEnd,int *cursor, FILE *pfdebug, int length)
+{
+	char	   *prefix = length == 4 ? "" : "#";
+	uint16		tmp2;
+	uint32		tmp4;
+	int			result;
+
+
+	if (*cursor + length > LogEnd)
+		return EOF;
+
+	switch(length)
+	{
+		case 2:
+			memcpy(&tmp2, buf + *cursor , 2);
+			result = (int) pg_ntoh16(tmp2);
+			*cursor += 2;
+			break;
+		case 4:
+			memcpy(&tmp4, buf + *cursor , 4);
+			result = (int) pg_ntoh32(tmp4);
+			*cursor += 4;
+			break;
+		default:
+			fprintf(pfdebug,
+			"integer of size %d not supported by pqTraceOutputInt", length);
+			return EOF;
+	}
+	fprintf(pfdebug, "%s%d", prefix, result);
+
+	if (*cursor < LogEnd)
+		fprintf(pfdebug, " ");
+	else
+		fprintf(pfdebug, "\n");
+
+	return (int) result;
+}
+
+/*
+ * pqTraceOutputString: output a null-terminated string to the log
+ */
+static int
+pqTraceOutputString(char *buf, int LogEnd,int *cursor, FILE *pfdebug)
+{
+	int		csr = *cursor;
+	int		slen;
+	char	*s;
+
+	while (csr < LogEnd && buf[csr])
+		csr++;
+
+	if (csr >= LogEnd)
+		return EOF;
+
+	slen = csr - *cursor;
+
+	s = (char *) malloc(sizeof(char) * (slen + 1));
+	memcpy(s, buf + *cursor, slen);
+	s[slen] = '\0';
+
+	fprintf(pfdebug, "\"%s\"", s);
+
+	*cursor = ++csr;
+
+	if (*cursor < LogEnd)
+		fprintf(pfdebug, " ");
+	else
+		fprintf(pfdebug, "\n");
+
+	return 0;
+}
+
+/*
+ * pqTraceOutputBinary: output a string possibly consisting of
+ * non-printable characters. Hex representation is used for such
+ * chars; others are printed normally.
+ *
+ * XXX this probably doesn't do a great job with multibyte chars, but then we
+ * don't know what is text and what encoding it'd be in.
+ */
+static void
+pqTraceOutputBinary(char *v, int length, FILE *pfdebug)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, pfdebug);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(char *buf, int LogEnd, int *cursor, FILE *pfdebug, int len)
+{
+	char	*v = '\0';
+
+	memcpy(v, buf + *cursor, len);
+	*cursor += len;
+
+	fprintf(pfdebug, "\'");
+	pqTraceOutputBinary(v, len, pfdebug);
+	fprintf(pfdebug, "\'");
+
+	if (*cursor < LogEnd)
+		fprintf(pfdebug, " ");
+	else
+		fprintf(pfdebug, "\n");
+}
+
+int
+pqTraceOutputMsg(PGconn *conn, int msgCursor, PGCommSource commsource)
+{
+	char		timestr[FORMATTED_TS_LEN];
+	char 		id = '\0';
+	uint32		length32;
+	int			length;
+	char	   *prefix = commsource == MSGDIR_FROM_BACKEND ? "<" : ">";
+	int			LogCount = conn->outCount;
+	int			LogEnd = commsource == MSGDIR_FROM_BACKEND ? conn->inCursor : conn->outMsgEnd;
+	int			LogCursor = msgCursor;
+	char		*logBuffer = commsource == MSGDIR_FROM_BACKEND ? conn->inBuffer : conn->outBuffer;
+	const char	*message_type = "UnkownMessage";
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	if (commsource == MSGDIR_FROM_BACKEND)
+		id = logBuffer[LogCursor++];
+	else
+	{
+		if (LogCursor > LogCount)
+			id = logBuffer[LogCount];
+	}
+
+	if (LogCursor + 4 > LogEnd)
+		return EOF;
+	memcpy(&length32, logBuffer + LogCursor , 4);
+	length = (int) pg_ntoh32(length32);
+	LogCursor += 4;
+
+	if (id == '\0')
+	{
+		switch(length)
+		{
+			case 8:
+			{
+				uint32		result32;
+				int			result;
+				memcpy(&result32, logBuffer + LogCursor, 4);
+				result = (int) pg_ntoh32(result32);
+				if (result == NEGOTIATE_SSL_CODE)
+					message_type = "SSLRequest";
+				else if (result == NEGOTIATE_GSS_CODE)
+					message_type = "GSSRequest";
+				break;
+			}
+			case 16:
+				message_type = "CancelRequest";
+				break;
+			default :
+				message_type = "StartupMessage";
+				break;
+		}
+	}
+	else
+		message_type =
+			pqGetProtocolMsgType(id, commsource);
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%s\t%d ", timestr, prefix, message_type, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			break;
+		case 'B':	/* Bind */
+		{
+			int nparams;
+			int nbytes;
+			int i;
+			pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			nparams = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+															conn->Pfdebug, 2);
+			for (i = 0; i < nparams; i++)
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+															conn->Pfdebug, 2);
+			nparams = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+															conn->Pfdebug, 2);
+			for (i = 0; i < nparams; i++)
+			{
+				nbytes = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+															conn->Pfdebug, 4);
+				pqTraceOutputNchar(logBuffer, LogEnd, &LogCursor,
+														conn->Pfdebug, nbytes);
+			}
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+			break;
+		}
+		case 'C':	/* Close(F) or Command Complete(B) */
+			if (commsource == MSGDIR_FROM_BACKEND)
+				fprintf(conn->Pfdebug, "\"%s\"\n", logBuffer + LogCursor);
+			else
+			{
+				pqTraceOutputByte1(LogEnd, &LogCursor, conn->Pfdebug,
+													logBuffer[LogCursor++]);
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			}
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			fprintf(conn->Pfdebug, "\n");
+			break;
+		case 'f':	/* Copy Fail */
+			fprintf(conn->Pfdebug, "\"%s\"\n", logBuffer + LogCursor);
+			pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputByte1(LogEnd, &LogCursor,
+									conn->Pfdebug, logBuffer[LogCursor++]);
+			while (LogEnd > LogCursor)
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (commsource == MSGDIR_FROM_BACKEND)
+			{
+				pqTraceOutputByte1(LogEnd, &LogCursor,
+										conn->Pfdebug, logBuffer[LogCursor++]);
+				while (LogEnd > LogCursor)
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+			}
+			else
+				fprintf(conn->Pfdebug, "\n");
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputByte1(LogEnd, &LogCursor,
+									conn->Pfdebug, logBuffer[LogCursor++]);
+			while (LogEnd > LogCursor)
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			if (commsource == MSGDIR_FROM_BACKEND)
+			{
+				int		len;
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+				while (LogEnd > LogCursor)
+				{
+					len = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+																conn->Pfdebug, 4);
+					pqTraceOutputNchar(logBuffer, LogEnd, &LogCursor,
+														conn->Pfdebug, len);
+				}
+			}
+			else
+			{
+				pqTraceOutputByte1(LogEnd, &LogCursor, conn->Pfdebug,
+													logBuffer[LogCursor++]);
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			}
+			break;
+		case 'E':	/* Execute(F) or Error Return(B) */
+			if (commsource == MSGDIR_FROM_BACKEND)
+			{
+				while (LogEnd > LogCursor)
+				{
+					pqTraceOutputByte1(LogEnd, &LogCursor, conn->Pfdebug,
+													logBuffer[LogCursor++]);
+					if (logBuffer[LogCursor] == '\0')
+						break;
+					pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+				}
+			}
+			else
+			{
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			}
+			break;
+		case 'F':	/* Function Call */
+			{
+				int nargs;
+				int nbytes;
+				int	i;
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+				nargs = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+																conn->Pfdebug, 2);
+				for (i = 0; i < nargs; i++)
+				{
+					nbytes = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+																conn->Pfdebug, 4);
+					pqTraceOutputNchar(logBuffer, LogEnd, &LogCursor,
+														conn->Pfdebug, nbytes);
+				}
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+				break;
+			}
+		case 'V':	/* Function Call response */
+			{
+				int		len;
+				len = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+				pqTraceOutputNchar(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, len);
+			break;
+			}
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			break;
+		case 'N':	/* Notice Response */
+			while (LogEnd > LogCursor)
+			{
+				pqTraceOutputByte1(LogEnd, &LogCursor, conn->Pfdebug,
+														logBuffer[LogCursor++]);
+				if (logBuffer[LogCursor] == '\0')
+					break;
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			}
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+			while (LogEnd > LogCursor)
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (commsource == MSGDIR_FROM_BACKEND)
+			{
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			}
+			else
+				fprintf(conn->Pfdebug, "\n");
+			break;
+		case 'P':	/* Parse */
+			{
+				int nparams;
+				int i;
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+				pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+				nparams = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor,
+																conn->Pfdebug, 2);
+				for (i = 0; i < nparams; i++)
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+				break;
+			}
+		case 'Q':	/* Query */
+			pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputByte1(LogEnd, &LogCursor, conn->Pfdebug,
+													logBuffer[LogCursor++]);
+			break;
+		case 'T':	/* Row Description */
+			{
+				int nfields;
+				int	i;
+				nfields = pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+				for (i = 0; i < nfields; i++)
+				{
+					pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+					pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 2);
+				}
+				break;
+			}
+		case '2':	/* Bind Complete */
+		case '3':	/* Close Complete */
+		case 'c':	/* Copy Done */
+		case 'I':	/* empty query */
+		case 'n':	/* No Data */
+		case '1':	/* Parse Complete */
+		case 's':	/* Portal Suspended */
+		case 'X':	/* Terminate */
+			/* No message content */
+			fprintf(conn->Pfdebug, "\n");
+			break;
+		case '\0' :
+			/* CancelRequest SSLRequest. GSSENCRequest and StartupMessage */
+			pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			if (length == 8)
+			{
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+				pqTraceOutputInt(logBuffer, LogEnd, &LogCursor, conn->Pfdebug, 4);
+			}
+			else
+			{
+				while (LogEnd > LogCursor)
+					pqTraceOutputString(logBuffer, LogEnd, &LogCursor,conn->Pfdebug);
+			}
+			break;
+		default:
+			fprintf(conn->Pfdebug, "Invalid Protocol:%c\n", id);
+			break;
+	}
+
+	return 0;
+}
diff --git a/src/interfaces/libpq/libpq-trace.h b/src/interfaces/libpq/libpq-trace.h
new file mode 100644
index 0000000..71b66da
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.h
@@ -0,0 +1,39 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-trace.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq protocol tracing.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-trace.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_TRACE_H
+#define LIBPQ_TRACE_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/* Log message source */
+typedef enum
+{
+	MSGDIR_FROM_BACKEND,
+	MSGDIR_FROM_FRONTEND
+} PGCommSource;
+
+extern int pqTraceOutputMsg(PGconn *conn, int msgCursor, PGCommSource commsource);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_TRACE_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..18a692d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1523,6 +1523,7 @@ PGAlignedXLogBlock
 PGAsyncStatusType
 PGCALL2
 PGChecksummablePage
+PGCommSource
 PGContextVisibility
 PGEvent
 PGEventConnDestroy
#142'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#141)
1 attachment(s)
Re: libpq debug log

On 2021-Mar-10, iwata.aya@fujitsu.com wrote:

Hi all,

Following all reviewer's advice, I have created a new patch.

In this patch, I add only two tracing entry points; I call pqTraceOutputMsg(PGconn conn, int msgCursor, PGCommSource commsource) in pqParseInput3 () and pqPutMsgEnd () to output log.
The argument contains message first byte offset called msgCursor because it is simpler to specify the buffer pointer in the caller.

In pqTraceOutputMsg(), the content common to all protocol messages (first timestamp, < or >, first 1 byte, and length) are output first and after that each protocol message contents is output. I referred to pqParseInput3 () , fe-exec.c and documentation page for that part.

This fix almost eliminates if(conn->Pfdebug) that was built into the code for other features.

This certainly looks much better! Thanks for getting it done so
quickly.

I didn't review the code super closely. I do see a compiler warning:

/pgsql/source/pipeline/src/interfaces/libpq/libpq-trace.c: In function 'pqTraceOutputNchar':
/pgsql/source/pipeline/src/interfaces/libpq/libpq-trace.c:373:2: warning: argument 1 null where non-null expected [-Wnonnull]
memcpy(v, buf + *cursor, len);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

and then when I try to run the "libpq_pipeline" program from the other
thread, I get a crash with this backtrace:

Program received signal SIGSEGV, Segmentation fault.
__memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:288
288 ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No existe el fichero o el directorio.
(gdb) bt
#0 __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:288
#1 0x00007ffff7f9b50b in pqTraceOutputNchar (buf=buf@entry=0x555555564660 "P",
LogEnd=LogEnd@entry=42, cursor=cursor@entry=0x7fffffffdeb4, pfdebug=0x555555569320, len=1)
at /pgsql/source/pipeline/src/interfaces/libpq/libpq-trace.c:373
#2 0x00007ffff7f9bc25 in pqTraceOutputMsg (conn=conn@entry=0x555555560260, msgCursor=<optimized out>,
commsource=commsource@entry=MSGDIR_FROM_FRONTEND)
at /pgsql/source/pipeline/src/interfaces/libpq/libpq-trace.c:476
#3 0x00007ffff7f96280 in pqPutMsgEnd (conn=conn@entry=0x555555560260)
at /pgsql/source/pipeline/src/interfaces/libpq/fe-misc.c:533
...

The attached patch fixes it, though I'm not sure that that's the final
form we want it to have (since we'd alloc and free repeatedly, making it
slow -- perhaps better to use a static, or ...? ).

--
�lvaro Herrera Valdivia, Chile
Essentially, you're proposing Kevlar shoes as a solution for the problem
that you want to walk around carrying a loaded gun aimed at your foot.
(Tom Lane)

Attachments:

fixsegv.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index ef294fa556..5d3b40a1d0 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -368,7 +368,15 @@ pqTraceOutputBinary(char *v, int length, FILE *pfdebug)
 static void
 pqTraceOutputNchar(char *buf, int LogEnd, int *cursor, FILE *pfdebug, int len)
 {
-	char	*v = '\0';
+	char	*v;
+
+	v = malloc(len);
+	if (v == NULL)
+	{
+		fprintf(pfdebug, "'..%d chars..'", len);
+		*cursor += len;
+		return;
+	}
 
 	memcpy(v, buf + *cursor, len);
 	*cursor += len;
@@ -377,6 +385,8 @@ pqTraceOutputNchar(char *buf, int LogEnd, int *cursor, FILE *pfdebug, int len)
 	pqTraceOutputBinary(v, len, pfdebug);
 	fprintf(pfdebug, "\'");
 
+	free(v);
+
 	if (*cursor < LogEnd)
 		fprintf(pfdebug, " ");
 	else
#143'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: 'alvherre@alvh.no-ip.org' (#142)
1 attachment(s)
Re: libpq debug log

I also found that DataRow messages are not being printed. This seems to
fix that in the normal case and singlerow, at least in pipeline mode.
Didn't verify the non-pipeline mode.

--
�lvaro Herrera 39�49'30"S 73�17'W
"Nunca se desea ardientemente lo que solo se desea por raz�n" (F. Alexandre)

Attachments:

datarow.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 68f0f6a081..e8db5edb71 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -869,6 +869,9 @@ getAnotherTuple(PGconn *conn, int msgLength)
 		goto advance_and_error;
 	}
 
+	if (conn->Pfdebug)
+		pqTraceOutputMsg(conn, conn->inStart, MSGDIR_FROM_BACKEND);
+
 	/* Advance inStart to show that the "D" message has been processed. */
 	conn->inStart = conn->inCursor;
 
#144'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: 'alvherre@alvh.no-ip.org' (#143)
Re: libpq debug log

On 2021-Mar-10, 'alvherre@alvh.no-ip.org' wrote:

I also found that DataRow messages are not being printed. This seems to
fix that in the normal case and singlerow, at least in pipeline mode.
Didn't verify the non-pipeline mode.

Hm, and RowDescription ('T') messages are also not being printed ... so
I think there's some larger problem here, and perhaps it needs to be
fixed using a different approach.

After staring at it a couple of times, I think that the places in
pqParseInput3() where there's a comment "... moves inStart itself" and
then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU
those are the places where the message in question would not reach the
"Successfully consumed this message" block that prints each message.

--
�lvaro Herrera 39�49'30"S 73�17'W
Maybe there's lots of data loss but the records of data loss are also lost.
(Lincoln Yeoh)

#145Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#144)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

After staring at it a couple of times, I think that the places in
pqParseInput3() where there's a comment "... moves inStart itself" and
then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU
those are the places where the message in question would not reach the
"Successfully consumed this message" block that prints each message.

Yeah, the whole business of just when a message has been "consumed"
is a stumbling block for libpq tracing. It was a real mess with the
existing code and v2 protocol, because we could actually try to parse
a message more than once, with the first few tries deciding that the
message wasn't all here yet (but nonetheless emitting partial trace
output).

Now that v2 is dead, the logic to abort because of the message not
being all arrived yet is basically useless: only the little bit of
code concerned with the message length word really needs to cope with
that. It's tempting to go through and get rid of all the now-unreachable
"return"s and such, but it seemed like it would be a lot of code churn for
not really that much gain.

I didn't look at the new version of the patch yet, so I'm not
sure whether the issues it still has are related to this.

regards, tom lane

#146Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#144)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

After staring at it a couple of times, I think that the places in
pqParseInput3() where there's a comment "... moves inStart itself" and
then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU
those are the places where the message in question would not reach the
"Successfully consumed this message" block that prints each message.

After digging around a little, I remember that there are a *bunch*
of places in fe-protocol3.c that advance inStart. The cleanest way
to shove this stuff in without rethinking that logic would be to
call pqTraceOutputMsg immediately before each such advance (using
the old inStart value as the message start address). A possible
advantage of doing it like that is that we'd be aware by that point
of whether we think the message is good or bad (or whether it was
good but we failed to process it, perhaps because of OOM). Seems
like that could be a useful thing to include in the log.

Or we could rethink the logic. It's not quite clear to me, after
all this time, why getRowDescriptions() et al are allowed to
move inStart themselves rather than letting the main loop in
pqParseInput3 do it. It might well be an artifact of having not
rewritten the v2 logic too much.

regards, tom lane

#147'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#145)
Re: libpq debug log

On 2021-Mar-10, Tom Lane wrote:

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

After staring at it a couple of times, I think that the places in
pqParseInput3() where there's a comment "... moves inStart itself" and
then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU
those are the places where the message in question would not reach the
"Successfully consumed this message" block that prints each message.

Yeah, the whole business of just when a message has been "consumed"
is a stumbling block for libpq tracing. It was a real mess with the
existing code and v2 protocol, because we could actually try to parse
a message more than once, with the first few tries deciding that the
message wasn't all here yet (but nonetheless emitting partial trace
output).

Hmm, that makes sense, but the issue I'm reporting is not the same,
unless I misunderstand you.

Now that v2 is dead, the logic to abort because of the message not
being all arrived yet is basically useless: only the little bit of
code concerned with the message length word really needs to cope with
that. It's tempting to go through and get rid of all the now-unreachable
"return"s and such, but it seemed like it would be a lot of code churn for
not really that much gain.

That sounds like an interesting exercise, and I bet it'd bring a lot of
code readability improvements.

I didn't look at the new version of the patch yet, so I'm not
sure whether the issues it still has are related to this.

The issues I noticed are related to separate messages rather than one
message split in pieces -- for example several DataRow messages are
processed internally in a loop, rather than each individually. The
number of places that need to be adjusted for things to AFAICT work
correctly are few enough; ISTM that the attached patch is sufficient.

(The business with the "logged" boolean is necessary so that we print to
the trace file any of those messages even if they are deviating from the
protocol.)

--
�lvaro Herrera Valdivia, Chile
"La experiencia nos dice que el hombre pel� millones de veces las patatas,
pero era forzoso admitir la posibilidad de que en un caso entre millones,
las patatas pelar�an al hombre" (Ijon Tichy)

#148'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#146)
Re: libpq debug log

On 2021-Mar-10, Tom Lane wrote:

Or we could rethink the logic. It's not quite clear to me, after
all this time, why getRowDescriptions() et al are allowed to
move inStart themselves rather than letting the main loop in
pqParseInput3 do it. It might well be an artifact of having not
rewritten the v2 logic too much.

I would certainly prefer that the logic stays put for the time being,
while I finalize the pipelining stuff.

--
�lvaro Herrera Valdivia, Chile

#149Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#146)
1 attachment(s)
Re: libpq debug log

I wrote:

Or we could rethink the logic. It's not quite clear to me, after
all this time, why getRowDescriptions() et al are allowed to
move inStart themselves rather than letting the main loop in
pqParseInput3 do it. It might well be an artifact of having not
rewritten the v2 logic too much.

After studying this further, I think we should apply the attached
patch to remove that responsibility from pqParseInput3's subroutines.
This will allow a single trace call near the bottom of pqParseInput3
to handle all cases that that function processes.

There are still some cowboy advancements of inStart in the functions
concerned with COPY processing, but it doesn't look like there's
much to be done about that. Those spots will need their own trace
calls.

BTW, while looking at this I concluded that getParamDescriptions
is actually buggy: if it's given a malformed message that seems
to need more data than the message length specifies, it just
returns EOF, resulting in an infinite loop. This function apparently
got missed while converting the logic from v2 (where that was the
right thing) to v3 (where it ain't). So that bit needs to be
back-patched. I'm tempted to back-patch the whole thing though.

regards, tom lane

Attachments:

consolidate-inStart-advancement.patchtext/x-diff; charset=us-ascii; name=consolidate-inStart-advancement.patchDownload
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 2ca8c057b9..233456fd90 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -290,8 +290,6 @@ pqParseInput3(PGconn *conn)
 						/* First 'T' in a query sequence */
 						if (getRowDescriptions(conn, msgLength))
 							return;
-						/* getRowDescriptions() moves inStart itself */
-						continue;
 					}
 					else
 					{
@@ -337,8 +335,7 @@ pqParseInput3(PGconn *conn)
 				case 't':		/* Parameter Description */
 					if (getParamDescriptions(conn, msgLength))
 						return;
-					/* getParamDescriptions() moves inStart itself */
-					continue;
+					break;
 				case 'D':		/* Data Row */
 					if (conn->result != NULL &&
 						conn->result->resultStatus == PGRES_TUPLES_OK)
@@ -346,8 +343,6 @@ pqParseInput3(PGconn *conn)
 						/* Read another tuple of a normal query response */
 						if (getAnotherTuple(conn, msgLength))
 							return;
-						/* getAnotherTuple() moves inStart itself */
-						continue;
 					}
 					else if (conn->result != NULL &&
 							 conn->result->resultStatus == PGRES_FATAL_ERROR)
@@ -462,7 +457,6 @@ handleSyncLoss(PGconn *conn, char id, int msgLength)
  * command for a prepared statement) containing the attribute data.
  * Returns: 0 if processed message successfully, EOF to suspend parsing
  * (the latter case is not actually used currently).
- * In the former case, conn->inStart has been advanced past the message.
  */
 static int
 getRowDescriptions(PGconn *conn, int msgLength)
@@ -577,9 +571,6 @@ getRowDescriptions(PGconn *conn, int msgLength)
 	/* Success! */
 	conn->result = result;
 
-	/* Advance inStart to show that the "T" message has been processed. */
-	conn->inStart = conn->inCursor;
-
 	/*
 	 * If we're doing a Describe, we're done, and ready to pass the result
 	 * back to the client.
@@ -603,9 +594,6 @@ advance_and_error:
 	if (result && result != conn->result)
 		PQclear(result);
 
-	/* Discard the failed message by pretending we read it */
-	conn->inStart += 5 + msgLength;
-
 	/*
 	 * Replace partially constructed result with an error result. First
 	 * discard the old result to try to win back some memory.
@@ -624,6 +612,12 @@ advance_and_error:
 	appendPQExpBuffer(&conn->errorMessage, "%s\n", errmsg);
 	pqSaveErrorResult(conn);
 
+	/*
+	 * Show the message as fully consumed, else pqParseInput3 will overwrite
+	 * our error with a complaint about that.
+	 */
+	conn->inCursor = conn->inStart + 5 + msgLength;
+
 	/*
 	 * Return zero to allow input parsing to continue.  Subsequent "D"
 	 * messages will be ignored until we get to end of data, since an error
@@ -635,12 +629,8 @@ advance_and_error:
 /*
  * parseInput subroutine to read a 't' (ParameterDescription) message.
  * We'll build a new PGresult structure containing the parameter data.
- * Returns: 0 if completed message, EOF if not enough data yet.
- * In the former case, conn->inStart has been advanced past the message.
- *
- * Note that if we run out of data, we have to release the partially
- * constructed PGresult, and rebuild it again next time.  Fortunately,
- * that shouldn't happen often, since 't' messages usually fit in a packet.
+ * Returns: 0 if processed message successfully, EOF to suspend parsing
+ * (the latter case is not actually used currently).
  */
 static int
 getParamDescriptions(PGconn *conn, int msgLength)
@@ -690,23 +680,16 @@ getParamDescriptions(PGconn *conn, int msgLength)
 	/* Success! */
 	conn->result = result;
 
-	/* Advance inStart to show that the "t" message has been processed. */
-	conn->inStart = conn->inCursor;
-
 	return 0;
 
 not_enough_data:
-	PQclear(result);
-	return EOF;
+	errmsg = libpq_gettext("insufficient data in \"t\" message");
 
 advance_and_error:
 	/* Discard unsaved result, if any */
 	if (result && result != conn->result)
 		PQclear(result);
 
-	/* Discard the failed message by pretending we read it */
-	conn->inStart += 5 + msgLength;
-
 	/*
 	 * Replace partially constructed result with an error result. First
 	 * discard the old result to try to win back some memory.
@@ -724,6 +707,12 @@ advance_and_error:
 	appendPQExpBuffer(&conn->errorMessage, "%s\n", errmsg);
 	pqSaveErrorResult(conn);
 
+	/*
+	 * Show the message as fully consumed, else pqParseInput3 will overwrite
+	 * our error with a complaint about that.
+	 */
+	conn->inCursor = conn->inStart + 5 + msgLength;
+
 	/*
 	 * Return zero to allow input parsing to continue.  Essentially, we've
 	 * replaced the COMMAND_OK result with an error result, but since this
@@ -737,7 +726,6 @@ advance_and_error:
  * We fill rowbuf with column pointers and then call the row processor.
  * Returns: 0 if processed message successfully, EOF to suspend parsing
  * (the latter case is not actually used currently).
- * In the former case, conn->inStart has been advanced past the message.
  */
 static int
 getAnotherTuple(PGconn *conn, int msgLength)
@@ -817,21 +805,14 @@ getAnotherTuple(PGconn *conn, int msgLength)
 		goto advance_and_error;
 	}
 
-	/* Advance inStart to show that the "D" message has been processed. */
-	conn->inStart = conn->inCursor;
-
 	/* Process the collected row */
 	errmsg = NULL;
 	if (pqRowProcessor(conn, &errmsg))
 		return 0;				/* normal, successful exit */
 
-	goto set_error_result;		/* pqRowProcessor failed, report it */
+	/* pqRowProcessor failed, fall through to report it */
 
 advance_and_error:
-	/* Discard the failed message by pretending we read it */
-	conn->inStart += 5 + msgLength;
-
-set_error_result:
 
 	/*
 	 * Replace partially constructed result with an error result. First
@@ -851,6 +832,12 @@ set_error_result:
 	appendPQExpBuffer(&conn->errorMessage, "%s\n", errmsg);
 	pqSaveErrorResult(conn);
 
+	/*
+	 * Show the message as fully consumed, else pqParseInput3 will overwrite
+	 * our error with a complaint about that.
+	 */
+	conn->inCursor = conn->inStart + 5 + msgLength;
+
 	/*
 	 * Return zero to allow input parsing to continue.  Subsequent "D"
 	 * messages will be ignored until we get to end of data, since an error
#150Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#149)
Re: libpq debug log

On 2021-Mar-10, Tom Lane wrote:

After studying this further, I think we should apply the attached
patch to remove that responsibility from pqParseInput3's subroutines.
This will allow a single trace call near the bottom of pqParseInput3
to handle all cases that that function processes.

Works for me.

BTW, while looking at this I concluded that getParamDescriptions
is actually buggy: if it's given a malformed message that seems
to need more data than the message length specifies, it just
returns EOF, resulting in an infinite loop. This function apparently
got missed while converting the logic from v2 (where that was the
right thing) to v3 (where it ain't). So that bit needs to be
back-patched.

Ah, that makes sense.

I'm tempted to back-patch the whole thing though.

+0.5 on that. I think we may be happy that all branches are alike
(though it doesn't look like this will cause any subtle bugs -- breakage
will be fairly obvious).

--
�lvaro Herrera 39�49'30"S 73�17'W
"Most hackers will be perfectly comfortable conceptualizing users as entropy
sources, so let's move on." (Nathaniel Smith)

#151Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#150)
Re: libpq debug log

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

On 2021-Mar-10, Tom Lane wrote:

After studying this further, I think we should apply the attached
patch to remove that responsibility from pqParseInput3's subroutines.
This will allow a single trace call near the bottom of pqParseInput3
to handle all cases that that function processes.

Works for me.

I dug into the git history a little and concluded that the reason for
doing that in the subroutines is that 92785dac2 made it so for this
reason:

+	/*
+	 * Advance inStart to show that the "D" message has been processed.  We
+	 * must do this before calling the row processor, in case it longjmps.
+	 */
+	conn->inStart = conn->inCursor;

We later gave up on allowing user-defined row processor callbacks,
but we failed to undo this messiness. Looking at what 92785dac2 did
confirms something else I'd been eyeing, which is why these subroutines
have their own checks for having consumed the right amount of data
instead of letting the master check in pqParseInput3 take care of it.
They didn't use to do that, but that check was hoisted up to before the
row processor call, so we wouldn't expose data from a corrupt message to
user code. So I think we can undo that too.

92785dac2 only directly hacked getRowDescriptions and getAnotherTuple,
but it looks like similar error handling was stuck into
getParamDescriptions later.

regards, tom lane

#152Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: 'alvherre@alvh.no-ip.org' (#142)
Re: libpq debug log

At Wed, 10 Mar 2021 10:31:27 -0300, "'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> wrote in

On 2021-Mar-10, iwata.aya@fujitsu.com wrote:

Hi all,

Following all reviewer's advice, I have created a new patch.

In this patch, I add only two tracing entry points; I call pqTraceOutputMsg(PGconn conn, int msgCursor, PGCommSource commsource) in pqParseInput3 () and pqPutMsgEnd () to output log.
The argument contains message first byte offset called msgCursor because it is simpler to specify the buffer pointer in the caller.

In pqTraceOutputMsg(), the content common to all protocol messages (first timestamp, < or >, first 1 byte, and length) are output first and after that each protocol message contents is output. I referred to pqParseInput3 () , fe-exec.c and documentation page for that part.

This fix almost eliminates if(conn->Pfdebug) that was built into the code for other features.

This certainly looks much better! Thanks for getting it done so
quickly.

I didn't review the code super closely. I do see a compiler warning:

+1 for the thanks for the quick work. I have some random comments
after a quick look on it.

The individual per-type-output functions are designed to fit to the
corresponding pqGet*() functions. On the other hand, now that we read
the message bytes knowing the exact format in advance as you did in
pqTraceOutputMsg(). So the complexity exist in the functions can be
eliminated by separating them into more type specific output
functions. For example, pqTraceOutputInt() is far simplified by
spliting it into pqTraceOutputInt16() and -32().

The output functions copy message bytes into local variable but the
same effect can be obtained by just casing via typed pointer type.

uint32 tmp4;
..
memcpy(&tmp4, buf + *cursor, 4);
result = (int) pg_ntoh32(tmp4);

can be written as

result = pg_ntoh32(* (uint32 *) (buf + *cursor));

I think we can get rid of copying in the output functions for String
and Bytes in different ways.

Now that we can manage our own reading pointer within
pqTraceOutputMsg(), the per-type-output functions can work only on the
reading pointer instead of buffer pointer and cursor, and length.
*I*'d want to make the output functions move the reading pointer by
themseves. pqTradeOutputMsg can be as simplified as the follows doing
this. End-of-message pointer may be useful to check read-overrunning
on the message buffer.

switch (id) {
case 'R':
pqTraceOutputInt32(&p, endptr, conn->Pfdebug);
fputc('\n', conn->Pfdebug);
break;
case 'K':
pqTraceOutputInt32(&p, endptr, conn->Pfdebug));
pqTraceOutputInt32(&p, endptr, conn->Pfdebug));
...

+	char	   *prefix = commsource == MSGDIR_FROM_BACKEND ? "<" : ">";
+	int			LogEnd = commsource == MSGDIR_FROM_BACKEND ? conn->inCursor : conn->outMsgEnd;
+	char		*logBuffer = commsource == MSGDIR_FROM_BACKEND ? conn->inBuffer 
..
+ 	if (commsource == MSGDIR_FROM_BACKEND)
+		id = logBuffer[LogCursor++];

Repeated usage of the ternaly operator on the same condition makes
code hard-to-read. It is simpler to consolidate them into one if-else
statement.

+ result = (int) pg_ntoh32(result32);
+ if (result == NEGOTIATE_SSL_CODE)

Maybe I'm missing something, but the above doesn't seem working. I
didn't see such message when making a SSL connection.

+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;

We can write that message out into tracing file.

+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;

Pfdebug is always NULL for V2 protocol there, so it can be an
assertion?

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#153Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kyotaro Horiguchi (#152)
Re: libpq debug log

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

Maybe I'm missing something, but the above doesn't seem working. I
didn't see such message when making a SSL connection.

As far as that goes, I don't see any point in trying to trace
connection-related messages, because there is no way to enable
tracing on a connection before it's created.

regards, tom lane

#154tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#152)
RE: libpq debug log

Alvaro-san, Tom-san, Horiguchi-san,

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

+1 for the thanks for the quick work. I have some random comments
after a quick look on it.

Thank you very much for giving many comments. And We're sorry to have caused you trouble. I told Iwata-san yesterday to modify the patch to use the logging function interface that Tom-san, Alvaro-san and I agreed on, that I'd review after the new revision has been posted, and let others know she is modifying the patch again so that they won't start reviewing. But I should have done the request on hackers.

We're catching up with your feedback and post the updated patch. Then I'll review it.

We appreciate your help so much.

Regards
Takayuki Tsunakawa

#155tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#152)
RE: libpq debug log

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

The output functions copy message bytes into local variable but the
same effect can be obtained by just casing via typed pointer type.

uint32 tmp4;
..
memcpy(&tmp4, buf + *cursor, 4);
result = (int) pg_ntoh32(tmp4);

can be written as

result = pg_ntoh32(* (uint32 *) (buf + *cursor));

I'm afraid we need to memcpy() because of memory alignment.

I think we can get rid of copying in the output functions for String
and Bytes in different ways.

I haven't looked at this code, but you sound right.

Regards
Takayuki Tsunakawa

#156Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Tom Lane (#153)
Re: libpq debug log

At Wed, 10 Mar 2021 20:38:20 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote in

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

Maybe I'm missing something, but the above doesn't seem working. I
didn't see such message when making a SSL connection.

As far as that goes, I don't see any point in trying to trace
connection-related messages, because there is no way to enable
tracing on a connection before it's created.

Yeah, agreed. In the previous version tracing functions are called
during protocol negotiation but that no longer happenes.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#157Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#154)
Re: libpq debug log

At Thu, 11 Mar 2021 01:51:55 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

Alvaro-san, Tom-san, Horiguchi-san,

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

+1 for the thanks for the quick work. I have some random comments
after a quick look on it.

Thank you very much for giving many comments. And We're sorry to
have caused you trouble. I told Iwata-san yesterday to modify the
patch to use the logging function interface that Tom-san, Alvaro-san
and I agreed on, that I'd review after the new revision has been

Yeah, I agreed at the time. Sorry for not responding it but had no
room in my mind to do that:p

posted, and let others know she is modifying the patch again so that
they won't start reviewing. But I should have done the request on
hackers.

We're catching up with your feedback and post the updated patch.
Then I'll review it.

We appreciate your help so much.

My pleasure.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#158Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#155)
Re: libpq debug log

At Thu, 11 Mar 2021 03:01:16 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

The output functions copy message bytes into local variable but the
same effect can be obtained by just casing via typed pointer type.

uint32 tmp4;
..
memcpy(&tmp4, buf + *cursor, 4);
result = (int) pg_ntoh32(tmp4);

can be written as

result = pg_ntoh32(* (uint32 *) (buf + *cursor));

I'm afraid we need to memcpy() because of memory alignment.

Right. So something like this?

unsigned char p;

p = buf + *cursor;
result = (uint32) (*p << 24) + (*(p + 1)) << 16 + ...);

I think we can get rid of copying in the output functions for String
and Bytes in different ways.

I haven't looked at this code, but you sound right.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#159tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#158)
RE: libpq debug log

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

Right. So something like this?

unsigned char p;

p = buf + *cursor;
result = (uint32) (*p << 24) + (*(p + 1)) << 16 + ...);

Yes, that would work (if p is a pointer), but I think memcpy() is enough like pqGetInt() does.

Regards
Takayuki Tsunakawa

#160Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#159)
Re: libpq debug log

At Thu, 11 Mar 2021 04:12:57 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

Right. So something like this?

unsigned char p;

p = buf + *cursor;
result = (uint32) (*p << 24) + (*(p + 1)) << 16 + ...);

Yes, that would work (if p is a pointer), but I think memcpy() is enough like pqGetInt() does.

On second thought, memcpy onto a local variable doesn't harm. I agreed
to you.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#161Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#150)
Re: libpq debug log

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

On 2021-Mar-10, Tom Lane wrote:

After studying this further, I think we should apply the attached
patch to remove that responsibility from pqParseInput3's subroutines.
This will allow a single trace call near the bottom of pqParseInput3
to handle all cases that that function processes.

Works for me.

Pushed that. I think we're now waiting on Iwata-san to finish a
new version of the tracing patch.

regards, tom lane

#162iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Tom Lane (#161)
1 attachment(s)
RE: libpq debug log

Alvaro san, Tom san Horiguchi san, Tsunakawa san and Kirk san,

Thank you very much for review and advice.

Works for me.

Pushed that. I think we're now waiting on Iwata-san to finish a new version of
the tracing patch.

Thank you very much Alvaro san and Tom san. Your patch and code change makes my work more smoothly.
And I am sorry for attaching not-good-work patch. fixsegv.patch help my implementation.

I update this patch to v25. I fix function arguments and something which are pointed out by reviewers.

I create protocol message reading function for each protocol message type. (Ex. pqTraceOutputB() read Bind message)
This makes the nesting shallower and makes the code easier to read.
Each data type output functions(Ex. pqTraceOutputString() ) are called from each protocol message type function.
Cursor operation is performed in each protocol message type function.

Like Kirk san, I share `make check` results. As far as I can see, tracing works well.
I compare this result with current master branch trace log. It does not miss/add any protocol message.
And Thank you Kirk san for your share of make check result and previous patch!

```
2021-03-15 08:10:44.288999 > Terminate 4
2021-03-15 08:10:44.317959 > Query 155 "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (some_nonexistent_parameter = true);"
2021-03-15 08:10:44.319121 < ErrorResponse 124 S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L "1447" R "parseRelOptionsInternal" \x00
2021-03-15 08:10:44.319161 < ReadyForQuery 5 I
2021-03-15 08:10:44.319221 > Query 144 "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (random_page_cost = 3.0);"
2021-03-15 08:10:44.320889 < CommandComplete 22 "CREATE TABLESPACE"
2021-03-15 08:10:44.320922 < ReadyForQuery 5 I
2021-03-15 08:10:44.320979 > Query 81 "SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';"
2021-03-15 08:10:44.324296 < RowDescription 35 #1 "spcoptions" 1213 #5 1009 #65535 -1 #0
2021-03-15 08:10:44.324335 < DataRow 32 #1 22 '{random_page_cost=3.0}'
2021-03-15 08:10:44.324347 < CommandComplete 13 "SELECT 1"
2021-03-15 08:10:44.324388 < ReadyForQuery 5 I
2021-03-15 08:10:44.324516 > Query 42 "DROP TABLESPACE regress_tblspacewith;"
2021-03-15 08:10:44.325423 < CommandComplete 20 "DROP TABLESPACE"
2021-03-15 08:10:44.325453 < ReadyForQuery 5 I
```

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Friday, March 5, 2021 3:17 PM

...

void pqLogMessage(PGconn *conn, const char *message, bool toServer);

Thank you Tsunakawa san. In v25 patch, I fixed qTraceOutputMessag() function to this style.
I am sorry changing arguments in previous patch. That was my misunderstanding.

From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Thursday, March 11, 2021 6:33 AM

...

There are still some cowboy advancements of inStart in the functions
concerned with COPY processing, but it doesn't look like there's much to be
done about that. Those spots will need their own trace calls.

I called pqTraceOutputMessage() from pqParseInput3(), COPY process and pqFunctionCall3(). pqFunctionCall3() is called by PQfn().
I found that there is protocol message id switch(). So I call the tracing function there.

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Thursday, March 11, 2021 10:20 AM

Thank you Horiguchi san. Your keen review suggestions and detailed advice. It is very helpful for updating my patch.

The individual per-type-output functions are designed to fit to the
corresponding pqGet*() functions. On the other hand, now that we read the
message bytes knowing the exact format in advance as you did in
pqTraceOutputMsg(). So the complexity exist in the functions can be
eliminated by separating them into more type specific output functions. For
example, pqTraceOutputInt() is far simplified by spliting it into
pqTraceOutputInt16() and -32().

Yes, I refer to pqGet functions. To make data type output function more simpler,
I separate output int function to pqTraceOutputInt16 and -32.

I think we can get rid of copying in the output functions for String and Bytes in
different ways.

I remove memcpy call from String and Bytes output function.
These functions just pass *message to fprintf().

+	/* Protocol 2.0 does not support tracing. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;

We can write that message out into tracing file.

+void
+PQtraceSetFlags(PGconn *conn, int flags) {
+	if (conn == NULL)
+		return;
+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;

Pfdebug is always NULL for V2 protocol there, so it can be an assertion?

Protocol v2.0 is not supported anymore. So I just remove code following;

+	/* Protocol 2.0 is not supported. */
+	if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)

I checked the libpq code to see if I could make this decision.
Code to check whether protocol version is 3 or less It had been removed from various place.

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Thursday, March 11, 2021 12:03 PM

...

At Wed, 10 Mar 2021 20:38:20 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote in

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

Maybe I'm missing something, but the above doesn't seem working. I
didn't see such message when making a SSL connection.

As far as that goes, I don't see any point in trying to trace
connection-related messages, because there is no way to enable tracing
on a connection before it's created.

Yeah, agreed. In the previous version tracing functions are called during
protocol negotiation but that no longer happenes.

I see. Thank you for your advice.
I remove these code from pqTraceOutputMessage() function.

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Thursday, March 11, 2021 1:31 PM

...

Yes, that would work (if p is a pointer), but I think memcpy() is enough like

pqGetInt() does.

On second thought, memcpy onto a local variable doesn't harm. I agreed to
you.

I am not change pqTraceOutputInt16(). But pqTraceOutputInt32() change like this;
+       memcpy(&result, data, 4);
+       result = (int) pg_ntoh32(result);

Regards,
Aya Iwata

Attachments:

v25-libpq-trace-log.patchapplication/octet-stream; name=v25-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 910e9a8..eb0c588 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -5941,12 +5941,28 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
+      Only available when protocol version 3 or higher is used.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message type, message length, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      For numerical quantities, 16-bit values are preceded with <literal>#</literal>
+      and 32-bit values are printed without a prefix.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -5961,6 +5977,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..2311399 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
@@ -113,6 +114,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
@@ -126,6 +128,7 @@ uninstall: uninstall-lib
 	rm -f '$(DESTDIR)$(includedir)/libpq-fe.h'
 	rm -f '$(DESTDIR)$(includedir)/libpq-events.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/libpq-int.h'
+	rm -f '$(DESTDIR)$(includedir_internal)/libpq-trace.h'
 	rm -f '$(DESTDIR)$(includedir_internal)/pqexpbuffer.h'
 	rm -f '$(DESTDIR)$(datadir)/pg_service.conf.sample'
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index bbc1f90..09f1111 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -179,3 +179,4 @@ PQgetgssctx               176
 PQsetSSLKeyPassHook_OpenSSL         177
 PQgetSSLKeyPassHook_OpenSSL         178
 PQdefaultSSLKeyPassHook_OpenSSL     179
+PQtraceSetFlags           180
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 4e21057..d6c5943 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6826,27 +6826,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 9a03804..5009e98 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -966,10 +966,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..ba682b3 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -49,6 +49,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "pg_config_paths.h"
 #include "port/pg_bswap.h"
@@ -84,9 +85,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +98,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +133,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +158,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +176,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +193,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +208,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +245,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +278,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +486,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +520,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +529,10 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug && (conn->outCount < conn->outMsgStart))
+		pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +959,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index eb55d52..da15337 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -28,6 +28,7 @@
 
 #include "libpq-fe.h"
 #include "libpq-int.h"
+#include "libpq-trace.h"
 #include "mb/pg_wchar.h"
 #include "port/pg_bswap.h"
 
@@ -414,6 +415,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1609,6 +1613,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2066,6 +2073,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index fa9b62a..59ea464 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -362,7 +362,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 2f052f6..9f79a7c 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -376,6 +376,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..3da0c91
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,810 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq-trace.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+static const char *pqGetProtocolMsgType(unsigned char c,
+										bool toServer);
+
+/* -------------------------
+ * FE/BE trace support
+ *
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceOutputFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.
+ * -------------------------
+ */
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * pqGetProtocolMsgType:
+ *		Get a protocol type from first byte identifier
+ */
+static const char *
+pqGetProtocolMsgType(unsigned char c, bool toServer)
+{
+	if (toServer == true &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	if (toServer == false &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	return "UnknownMessage:";
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(const char *data, FILE *pfdebug)
+{
+	fputc(' ', pfdebug);
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*data))
+		fprintf(pfdebug, "\\x%02x", *data);
+	else
+		fprintf(pfdebug, "%c", *data);
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(const char *data, FILE *pfdebug)
+{
+	uint16		tmp2;
+	int			result;
+
+	memcpy(&tmp2, data , 2);
+	result = (int) pg_ntoh16(tmp2);
+	fputc(' ', pfdebug);
+	fprintf(pfdebug, "#%d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(const char *data, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data, 4);
+	result = (int) pg_ntoh32(result);
+	fputc(' ', pfdebug);
+	fprintf(pfdebug, "%d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(const char *data, FILE *pfdebug)
+{
+	fputc(' ', pfdebug);
+	fprintf(pfdebug, "\"%s\"", data);
+}
+
+/*
+ * pqTraceOutputBinary: output a string possibly consisting of
+ * non-printable characters. Hex representation is used for such
+ * chars; others are printed normally.
+ *
+ * XXX this probably doesn't do a great job with multibyte chars, but then we
+ * don't know what is text and what encoding it'd be in.
+ */
+static void
+pqTraceOutputBinary(const char *v, int length, FILE *pfdebug)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, pfdebug);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(const char *data, FILE *pfdebug, int len)
+{
+	fputc(' ', pfdebug);
+	fprintf(pfdebug, "\'");
+	pqTraceOutputBinary(data, len, pfdebug);
+	fprintf(pfdebug, "\'");
+}
+
+static int
+moveStrCursor(const char *buf, int cursor, int end)
+{
+	while (cursor < end && buf[cursor])
+		cursor++;
+	cursor++;
+
+	return cursor;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(const char *message, FILE *f)
+{
+	pqTraceOutputInt32(message, f);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt32(message + cursor, f);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(const char *message, int end, FILE *f)
+{
+	int cursor = 0;
+	int nparams;
+	int nbytes;
+	int i;
+
+	pqTraceOutputString(message + cursor, f);
+	cursor = moveStrCursor(message, cursor, end);
+	pqTraceOutputString(message + cursor, f);
+	cursor = moveStrCursor(message, cursor, end);
+	nparams = pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	for (i = 0; i < nparams; i++)
+	{
+		pqTraceOutputInt16(message + cursor, f);
+		cursor += 2;
+	}
+
+	nparams = pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message + cursor, f);
+		cursor += 4;
+		if (nbytes == -1)
+			break;
+		pqTraceOutputNchar(message + cursor, f, nbytes);
+		cursor += nbytes;
+	}
+
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt16(message + cursor, f);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(const char *message, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputByte1(message + cursor, f);
+		cursor++;
+		pqTraceOutputString(message + cursor, f);
+	}
+	else
+		pqTraceOutputString(message + cursor, f);
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	pqTraceOutputString(message, f);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message + cursor, f);
+	cursor++;
+
+	while (end > cursor)
+	{
+		pqTraceOutputInt16(message + cursor, f);
+		cursor += 2;
+	}
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (!toServer)
+	{
+		pqTraceOutputByte1(message + cursor, f);
+		cursor++;
+
+		while (end > cursor)
+		{
+			pqTraceOutputInt16(message + cursor, f);
+			cursor += 2;
+		}
+	}
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message + cursor, f);
+	cursor++;
+
+	while (end > cursor)
+	{
+		pqTraceOutputInt16(message + cursor, f);
+		cursor += 2;
+	}
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(const char *message, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputByte1(message + cursor, f);
+		cursor++;
+		pqTraceOutputString(message + cursor, f);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		nfields = pqTraceOutputInt16(message + cursor, f);
+		cursor += 2;
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message + cursor, f);
+			cursor += 4;
+			if (len == -1)
+				break;
+			pqTraceOutputNchar(message + cursor, f, len);
+			cursor += len;
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputString(message + cursor, f);
+		cursor = moveStrCursor(message, cursor, end);
+		pqTraceOutputInt32(message + cursor, f);
+	}
+	else
+	{
+		while (end > cursor)
+		{
+			pqTraceOutputByte1(message + cursor, f);
+			if (message[cursor] == '\0')
+				break;
+			cursor++;
+			pqTraceOutputString(message + cursor, f);
+			cursor = moveStrCursor(message, cursor, end);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int nargs;
+	int nbytes;
+	int	i;
+
+	pqTraceOutputInt32(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	nargs = pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	for (i = 0; i < nargs; i++)
+	{
+		nbytes = pqTraceOutputInt32(message + cursor, f);
+		cursor += 4;
+		if (nbytes == -1)
+			break;
+		pqTraceOutputNchar(message + cursor, f, nbytes);
+		cursor += nbytes;
+	}
+
+	pqTraceOutputInt16(message + cursor, f);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int		len;
+
+	len = pqTraceOutputInt32(message + cursor, f);
+	cursor += 4;
+	if ( len != -1)
+		pqTraceOutputNchar(message + cursor, f, len);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt32(message + cursor, f);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	while (end > cursor)
+	{
+		pqTraceOutputByte1(message, f);
+		cursor++;
+		if (message[cursor] == '\0')
+			break;
+		pqTraceOutputString(message + cursor, f);
+		cursor = moveStrCursor(message, cursor, end);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+	pqTraceOutputString(message + cursor, f);
+	cursor = moveStrCursor(message, cursor, end);
+	pqTraceOutputString(message + cursor, f);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputInt32(message + cursor, f);
+		cursor += 4;
+	}
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (!toServer)
+	{
+		pqTraceOutputString(message + cursor, f);
+		cursor = moveStrCursor(message, cursor, end);
+		pqTraceOutputString(message + cursor, f);
+	}
+}
+
+/* Parse */
+static void
+pqTraceOutputP(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nparams;
+	int i;
+
+	pqTraceOutputString(message + cursor, f);
+	cursor = moveStrCursor(message, cursor, end);
+	pqTraceOutputString(message + cursor, f);
+	cursor = moveStrCursor(message, cursor, end);
+	nparams = pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	for (i = 0; i < nparams; i++)
+	{
+		pqTraceOutputInt32(message + cursor, f);
+		cursor += 4;
+	}
+}
+
+/* Query */
+static void
+pqTraceOutputQ(const char *message, FILE *f)
+{
+	pqTraceOutputString(message, f);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(const char *message, FILE *f)
+{
+	pqTraceOutputByte1(message, f);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message + cursor, f);
+		cursor = moveStrCursor(message, cursor, end);
+		pqTraceOutputInt32(message + cursor, f);
+		cursor +=4;
+		pqTraceOutputInt16(message + cursor, f);
+		cursor +=2;
+		pqTraceOutputInt32(message + cursor, f);
+		cursor +=4;
+		pqTraceOutputInt16(message + cursor, f);
+		cursor +=2;
+		pqTraceOutputInt32(message + cursor, f);
+		cursor +=4;
+		pqTraceOutputInt16(message + cursor, f);
+		cursor +=2;
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+	char		timestr[128];
+	char 		id = '\0';
+	int			length;
+	char	   *prefix = toServer ? ">" : "<";
+	int			LogCursor = 0;
+	int			LogEnd;
+	const char	*message_type = "UnkownMessage";
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
+		gettimeofday(&tval, NULL);
+		stamp_time = (pg_time_t) tval.tv_sec;
+
+		strftime(timestr,  sizeof(timestr),
+				 "%Y-%m-%d %H:%M:%S",
+				 localtime(&stamp_time));
+		/* append microseconds */
+		sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+	}
+	else
+		timestr[0] = '\0';
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+
+	message_type = pqGetProtocolMsgType(id, toServer);
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%s\t%d", timestr, prefix, message_type, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(message + LogCursor, conn->Pfdebug, toServer);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(message + LogCursor, conn->Pfdebug, toServer);
+			break;
+		case 'E':	/* Execute(F) or Error Return(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case '2':	/* Bind Complete */
+		case '3':	/* Close Complete */
+		case 'c':	/* Copy Done */
+		case 'I':	/* empty query */
+		case 'n':	/* No Data */
+		case '1':	/* Parse Complete */
+		case 's':	/* Portal Suspended */
+		case 'X':	/* Terminate */
+			/* No message content */
+			break;
+		default:
+			fprintf(conn->Pfdebug, "Invalid Protocol:%c\n", id);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
diff --git a/src/interfaces/libpq/libpq-trace.h b/src/interfaces/libpq/libpq-trace.h
new file mode 100644
index 0000000..6a643db
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.h
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * libpq-trace.h
+ *	  This file contains definitions for structures and
+ *	  externs for functions used by libpq protocol tracing.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/interfaces/libpq/libpq-trace.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef LIBPQ_TRACE_H
+#define LIBPQ_TRACE_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif							/* LIBPQ_TRACE_H */
#163tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#162)
RE: libpq debug log

I'm looking at the last file libpq-trace.c. I'll continue the review after lunch. Below are some comments so far.

(1)
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
+      Only available when protocol version 3 or higher is used.

The last line is unnecessary now that v2 is not supported.

(2)
@@ -113,6 +114,7 @@ install: all installdirs install-lib
 	$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
 	$(INSTALL_DATA) $(srcdir)/libpq-int.h '$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h '$(DESTDIR)$(includedir_internal)'
 	$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample '$(DESTDIR)$(datadir)/pg_service.conf.sample'

Why is this necessary?

(3) libpq-trace.h
+#ifdef __cplusplus
+extern "C"
+{

This is unnecessary because pqTraceOutputMessage() is for libpq's internal use. This extern is for the user's C++ application to call public libpq functions.

+#include "libpq-fe.h"
+#include "libpq-int.h"

I think these can be removed by declaring the struct and function as follows:

struct pg_conn;
extern void pqTraceOutputMessage(struct pg_conn *conn, const char *message, bool toServer);

But... after doing the above, only this declaration is left in libpq-trace.h. Why don't you put your original declaration using PGconn in libpq-int.h and remove libpq-trace.h?

(4)
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug && (conn->outCount < conn->outMsgStart))
+		pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);

() surrounding the condition after && can be removed.

(5)
+static const char *pqGetProtocolMsgType(unsigned char c,
+										bool toServer);

This is unnecessary because the function definition precedes its only one call site.

(6)
+ * We accumulate frontend message pieces in an array as the libpq code writes
+ * them, and log the complete message when pqTraceOutputFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the server.

The first paragraph is no longer true. I think this entire comment is unnecessary.

(7)
+static const char *
+pqGetProtocolMsgType(unsigned char c, bool toServer)
+{
+	if (toServer == true &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	if (toServer == false &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	return "UnknownMessage:";
+}

"== true/false" can be removed. libpq doesn't appear to use such style.

Why don't we embed this processing in pqTraceOutputMessage() because this function is short and called only once? The added benefit would be that you can print an invalid message type byte like this:

fprintf(..., "Unknown message: %02x\n", ...);

Regards
Takayuki Tsunakawa

#164tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#162)
RE: libpq debug log

I've not finished reviewing yet, but there seems to be many mistakes. I'm sending second set of review comments now so you can fix them in parallel.

(8)
+ char id = '\0';

This initialization is not required because id will always be assigned a value shortly.

(9)
+static int
+pqTraceOutputInt32(const char *data, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data, 4);
+	result = (int) pg_ntoh32(result);
+	fputc(' ', pfdebug);
+	fprintf(pfdebug, "%d", result);
+
+	return result;
+}

fputc() and fprintf() can be merged into one fprintf() call for efficiency.

TBH, it feels strange that an "output" function returns a value. But I understood this as a positive compromise for reducing code; without this, the caller has to do memcpy() and endianness conversion.

(10)
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt32(message + cursor, f);
+}

I don't think you need to always use a cursor variable in order to align with more complex messages. That is, you can just write:

+ pqTraceOutputInt32(message, f);
+ pqTraceOutputInt32(message + 4, f);

(11)
+		default:
+			fprintf(conn->Pfdebug, "Invalid Protocol:%c\n", id);
+			break;
+

(This is related to (7))
You can remove this default label if you exit the function before the switch statement when the message type is unknown. Make sure to output \n in that case.

(12) pqTraceOutputB
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message + cursor, f);
+		cursor += 4;
+		if (nbytes == -1)
+			break;
+		pqTraceOutputNchar(message + cursor, f, nbytes);
+		cursor += nbytes;
+	}

Not break but continue, because non-NULL parameters may follow a NULL parameter.

(13) pqTraceOutputB
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt16(message + cursor, f);
+}

This part doesn't seem to match the following description.

----------
After the last parameter, the following fields appear:

Int16
The number of result-column format codes that follow (denoted R below). This can be zero to indicate that there are no result columns or that the result columns should all use the default format (text); or one, in which case the specified format code is applied to all result columns (if any); or it can equal the actual number of result columns of the query.

Int16[R]
The result-column format codes. Each must presently be zero (text) or one (binary).
----------

(14)
The processing for CancelRequest message is missing?

(15)
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message + cursor, f);
+	cursor++;
+
+	while (end > cursor)
+	{
+		pqTraceOutputInt16(message + cursor, f);
+		cursor += 2;
+	}
+}
+

According to the following description, for loop should be used.

----------
Int16
The number of columns in the data to be copied (denoted N below).

Int16[N]
The format codes to be used for each column. Each must presently be zero (text) or one (binary). All must be zero if the overall copy format is textual.
----------

Regards
Takayuki Tsunakawa

#165tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#162)
RE: libpq debug log

I've finished the review. Here are the last set of comments.

(16)
(15) is also true for the processing of 'H' message.

(17) pqTraceOutputD
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message + cursor, f);
+			cursor += 4;
+			if (len == -1)
+				break;
+			pqTraceOutputNchar(message + cursor, f, len);
+			cursor += len;
+		}

Not break but continue, because non-NULL columns may follow a NULL column.

(18)
+		case 'E':	/* Execute(F) or Error Return(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;

Error Return -> ErrorResponse

(19) pqTraceOutputF
Check the protocol again. Two for loops should be required, one for format codes and another for arguments.

(20)
+ if ( len != -1)

Remove extra space before len.

(21)
I guess you intentionally omitted the following messages because they are only used during connection establishment. I'm fine with it. I wrote this just in case you missed them.

GSSENCRequest (F)
Int32(8)

GSSResponse (F)
Byte1('p')

PasswordMessage (F)
Byte1('p')

SASLInitialResponse (F)
Byte1('p')

SASLResponse (F)
Byte1('p')

(22)
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+	pqTraceOutputString(message + cursor, f);

Not Int16 but Int32.

Regards
Takayuki Tsunakawa

#166Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#162)
Re: libpq debug log

On 2021-Mar-15, iwata.aya@fujitsu.com wrote:

I create protocol message reading function for each protocol message type. (Ex. pqTraceOutputB() read Bind message)
This makes the nesting shallower and makes the code easier to read.

I'm not sure I agree with this structural change. Yes, it is less
indentation, but these functions are all quite short and simple anyway.
But I don't disagree strongly with it either.

Four issues:
* There's no cross-check that the protocol message length word
corresponds to what we actually print. I think one easy way to
cross-check that would be to return the "count" from the type-specific
routines, and verify that it matches "length" in pqTraceOutputMessage.
(If the separate routines are removed and the code put back in one
giant switch, then the "count" is already available when the switch
ends.)

* If we have separate functions for each message type, then we can have
that function supply the message name by itself, rather than have the
separate protocol_message_type_f / _b arrays that print it.

* If we make each type-specific function print the message name, then we
need to make the same function print the message length, because it
comes after. So the function would have to receive the length (so
that it can be printed). But I think we should still have the
cross-check I mentioned in my first point occur in the main
pqTraceOutputMessage, not the type-specific function, for fear that we
will forget the cross-check in one of the functions and never realize
that we did.

* I would make the pqTraceOutputInt16() function and siblings advance
the cursor themselves, actually. I think something like this:
static int
pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
{
uint16 tmp;
int result;

memcpy(&tmp, data + *cursor, 2);
*cursor += 2;
result = (int) pg_ntoh16(tmp);
fprintf(pfdebug, " #%d", result);

return result;
}
(So the caller has to pass the original "data" pointer, not
"data+cursor"). This means the caller no longer has to do "cursor+=N"
at each place. Also, you get rid of the moveStrCursor() which does
not look good.

Bikeshedding item:
* I'm not fond of the idea of prefixing "#" for 16bit ints and no
prefix for 32bit ints. Seems obscure and the output looks weird.
I would use a one-letter prefix for each type, "w" for 32-bit ints
(stands for "word") and "h" for 16-bit ints (stands for "half-word").
Message length goes unadorned. Then you'd have lines like this

2021-03-15 08:10:44.324296 < RowDescription 35 h1 "spcoptions" w1213 h5 w1009 h65535 w-1 h0
2021-03-15 08:10:44.324335 < DataRow 32 h1 w22 '{random_page_cost=3.0}'

* I don't like that pqTraceOutputS/H print nothing when !toServer. I
think they should complain.

--
�lvaro Herrera 39�49'30"S 73�17'W

#167tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Alvaro Herrera (#166)
RE: libpq debug log

Alvaro-san,

Thank you for taking your time to take a look at an incomplete patch. I thought I would ask you for final check for commit after Iwata-san has reflected my review comments.

I discussed with Iwata-sn your below comments. Let me convey her opinions. (She is now focusing on fixing the patch.) We'd appreciate if you could tweak her completed patch.

From: Alvaro Herrera <alvherre@alvh.no-ip.org>

* There's no cross-check that the protocol message length word
corresponds to what we actually print. I think one easy way to
cross-check that would be to return the "count" from the type-specific
routines, and verify that it matches "length" in pqTraceOutputMessage.
(If the separate routines are removed and the code put back in one
giant switch, then the "count" is already available when the switch
ends.)

We don't think the length check is necessary, because 1) for FE->BE, correct messages are always assembled, and 2) for BE->FE, the parsing and decomposition of messages have already checked the messages.

* If we have separate functions for each message type, then we can have
that function supply the message name by itself, rather than have the
separate protocol_message_type_f / _b arrays that print it.

I felt those two arrays are clumsy and thought of suggesting to remove them. But I didn't because the functions or case labels for each message type have to duplicate the printing of header fields: timestamp, message type, and length. Maybe we can change the order of output to "timestamp length message_type content", but I kind of prefer the current order. What do you think? (I can understand removing the clumsy static arrays should be better than the order of output fields.)

* If we make each type-specific function print the message name, then we
need to make the same function print the message length, because it
comes after. So the function would have to receive the length (so
that it can be printed). But I think we should still have the
cross-check I mentioned in my first point occur in the main
pqTraceOutputMessage, not the type-specific function, for fear that we
will forget the cross-check in one of the functions and never realize
that we did.

As mentioned above, we think the current structure is better for smaller and readable code.

* I would make the pqTraceOutputInt16() function and siblings advance
the cursor themselves, actually. I think something like this:
static int
pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
{
uint16 tmp;
int result;

memcpy(&tmp, data + *cursor, 2);
*cursor += 2;
result = (int) pg_ntoh16(tmp);
fprintf(pfdebug, " #%d", result);

return result;
}
(So the caller has to pass the original "data" pointer, not
"data+cursor"). This means the caller no longer has to do "cursor+=N"
at each place. Also, you get rid of the moveStrCursor() which does
not look good.

That makes sense, because in fact the patch mistakenly added 4 when it should add 2. Also, the code would become smaller.

* I'm not fond of the idea of prefixing "#" for 16bit ints and no
prefix for 32bit ints. Seems obscure and the output looks weird.
I would use a one-letter prefix for each type, "w" for 32-bit ints
(stands for "word") and "h" for 16-bit ints (stands for "half-word").
Message length goes unadorned. Then you'd have lines like this

2021-03-15 08:10:44.324296 < RowDescription 35 h1 "spcoptions"
w1213 h5 w1009 h65535 w-1 h0
2021-03-15 08:10:44.324335 < DataRow 32 h1 w22
'{random_page_cost=3.0}'

Yes, actually I felt something similar. Taking a second thought, I think we don't have to have any prefix because it doesn't help users. So we're removing the prefix. We don't have any opinion on adding some prefix.

* I don't like that pqTraceOutputS/H print nothing when !toServer. I
think they should complain.

Yes, the caller should not call the function if there's no message content. That way, the function doesn't need the toServer argument.

Regards
Takayuki Tsunakawa

#168Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#167)
Re: libpq debug log

At Wed, 17 Mar 2021 02:09:32 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

Alvaro-san,

Thank you for taking your time to take a look at an incomplete patch. I thought I would ask you for final check for commit after Iwata-san has reflected my review comments.

I discussed with Iwata-sn your below comments. Let me convey her opinions. (She is now focusing on fixing the patch.) We'd appreciate if you could tweak her completed patch.

From: Alvaro Herrera <alvherre@alvh.no-ip.org>

* There's no cross-check that the protocol message length word
corresponds to what we actually print. I think one easy way to
cross-check that would be to return the "count" from the type-specific
routines, and verify that it matches "length" in pqTraceOutputMessage.
(If the separate routines are removed and the code put back in one
giant switch, then the "count" is already available when the switch
ends.)

We don't think the length check is necessary, because 1) for FE->BE, correct messages are always assembled, and 2) for BE->FE, the parsing and decomposition of messages have already checked the messages.

Maybe it is not for the sanity of message bytes but to check if the
logging-stuff is implemented in sync with the messages structure. If
the given message length differs from the length the logging facility
read after scanning a message bytes, it's sign of something wrong in
*the logging feature*.

* If we have separate functions for each message type, then we can have
that function supply the message name by itself, rather than have the
separate protocol_message_type_f / _b arrays that print it.

I felt those two arrays are clumsy and thought of suggesting to remove them. But I didn't because the functions or case labels for each message type have to duplicate the printing of header fields: timestamp, message type, and length. Maybe we can change the order of output to "timestamp length message_type content", but I kind of prefer the current order. What do you think? (I can understand removing the clumsy static arrays should be better than the order of output fields.)

+1 for removing the arrays.

* If we make each type-specific function print the message name, then we
need to make the same function print the message length, because it
comes after. So the function would have to receive the length (so
that it can be printed). But I think we should still have the
cross-check I mentioned in my first point occur in the main
pqTraceOutputMessage, not the type-specific function, for fear that we
will forget the cross-check in one of the functions and never realize
that we did.

As mentioned above, we think the current structure is better for smaller and readable code.

* I would make the pqTraceOutputInt16() function and siblings advance
the cursor themselves, actually. I think something like this:
static int
pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
{
uint16 tmp;
int result;

memcpy(&tmp, data + *cursor, 2);
*cursor += 2;
result = (int) pg_ntoh16(tmp);
fprintf(pfdebug, " #%d", result);

return result;
}
(So the caller has to pass the original "data" pointer, not
"data+cursor"). This means the caller no longer has to do "cursor+=N"
at each place. Also, you get rid of the moveStrCursor() which does
not look good.

That makes sense, because in fact the patch mistakenly added 4 when it should add 2. Also, the code would become smaller.

FWIW, that's what I suggested upthread:p So +3.

me> *I*'d want to make the output functions move the reading pointer by
me> themseves. pqTradeOutputMsg can be as simplified as the follows doing

* I'm not fond of the idea of prefixing "#" for 16bit ints and no
prefix for 32bit ints. Seems obscure and the output looks weird.
I would use a one-letter prefix for each type, "w" for 32-bit ints
(stands for "word") and "h" for 16-bit ints (stands for "half-word").
Message length goes unadorned. Then you'd have lines like this

2021-03-15 08:10:44.324296 < RowDescription 35 h1 "spcoptions"
w1213 h5 w1009 h65535 w-1 h0
2021-03-15 08:10:44.324335 < DataRow 32 h1 w22
'{random_page_cost=3.0}'

Yes, actually I felt something similar. Taking a second thought, I think we don't have to have any prefix because it doesn't help users. So we're removing the prefix. We don't have any opinion on adding some prefix.

It would help when the value is "255", which is confusing between -1
(or 255) in byte or 255 in 2-byte word. Or when the value looks like
broken. I'd suggest "b"yte for 1 byte, "s"hort for 2 bytes, "l"ong for
4 bytes ('l' is confusing with '1', but anyway it is not needed)..

* I don't like that pqTraceOutputS/H print nothing when !toServer. I
think they should complain.

Yes, the caller should not call the function if there's no message content. That way, the function doesn't need the toServer argument.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#169iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#163)
1 attachment(s)
RE: libpq debug log

Hi Tsunakawa san, Alvaro san,

Thank you very much for your review. It helped me a lot to make the patch better.
I update patch to v26.
This patch has been updated according to Tsunakawa san and Alvaro san review comments.

The output is following;
```
2021-03-17 14:43:16.411238 > Terminate 4
2021-03-17 14:43:16.441775 > Query 155 "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (some_nonexistent_parameter = true);"
2021-03-17 14:43:16.442935 < ErrorResponse 124 S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L "1447" R "parseRelOptionsInternal" \x00 "Z"
2021-03-17 14:43:16.442977 < ReadyForQuery 5 I
2021-03-17 14:43:16.443042 > Query 144 "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (random_page_cost = 3.0);"
2021-03-17 14:43:16.444774 < CommandComplete 22 "CREATE TABLESPACE"
2021-03-17 14:43:16.444807 < ReadyForQuery 5 I
2021-03-17 14:43:16.444878 > Query 81 "SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';"
2021-03-17 14:43:16.448117 < RowDescription 35 1 "spcoptions" 1213 5 1009 65535 -1 0
2021-03-17 14:43:16.448157 < DataRow 32 1 22 '{random_page_cost=3.0}'
2021-03-17 14:43:16.448171 < CommandComplete 13 "SELECT 1"
```

-----Original Message-----
From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Tuesday, March 16, 2021 12:03 PM

(1)
-      Enables  tracing of the client/server communication to a debugging
file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
+      Only available when protocol version 3 or higher is used.

The last line is unnecessary now that v2 is not supported.

I removed last comment from documentation file.

(2)
@@ -113,6 +114,7 @@ install: all installdirs install-lib
$(INSTALL_DATA) $(srcdir)/libpq-fe.h '$(DESTDIR)$(includedir)'
$(INSTALL_DATA) $(srcdir)/libpq-events.h '$(DESTDIR)$(includedir)'
$(INSTALL_DATA) $(srcdir)/libpq-int.h
'$(DESTDIR)$(includedir_internal)'
+	$(INSTALL_DATA) $(srcdir)/libpq-trace.h
'$(DESTDIR)$(includedir_internal)'
$(INSTALL_DATA) $(srcdir)/pqexpbuffer.h
'$(DESTDIR)$(includedir_internal)'
$(INSTALL_DATA) $(srcdir)/pg_service.conf.sample
'$(DESTDIR)$(datadir)/pg_service.conf.sample'

Why is this necessary?

I think it is not necessary. I removed it.

(3) libpq-trace.h
+#ifdef __cplusplus
+extern "C"
+{

This is unnecessary because pqTraceOutputMessage() is for libpq's internal
use. This extern is for the user's C++ application to call public libpq
functions.

+#include "libpq-fe.h"
+#include "libpq-int.h"

I think these can be removed by declaring the struct and function as follows:

struct pg_conn;
extern void pqTraceOutputMessage(struct pg_conn *conn, const char
*message, bool toServer);

But... after doing the above, only this declaration is left in libpq-trace.h. Why
don't you put your original declaration using PGconn in libpq-int.h and remove
libpq-trace.h?

I remove this file.
I was wondering whether to delete this file during the development of v25 patch. I leave it because it keep scalability.
If tracing process become update and it have a lot of extern function, leave this header file is meaningful.
However I think it does not happen. So I remove it.

(4)
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug && (conn->outCount < conn->outMsgStart))
+		pqTraceOutputMessage(conn, conn->outBuffer +
conn->outCount, true);

() surrounding the condition after && can be removed.

I removed this (). And This if () statement has been modified according to the review comment (14).

(5)
+static const char *pqGetProtocolMsgType(unsigned char c,
+
bool toServer);

This is unnecessary because the function definition precedes its only one call
site.

Yes, I removed this definition.

(6)
+ * We accumulate frontend message pieces in an array as the libpq code
+ writes
+ * them, and log the complete message when pqTraceOutputFeMsg is called.
+ * For backend, we print the pieces as soon as we receive them from the
server.

The first paragraph is no longer true. I think this entire comment is
unnecessary.

I removed this explanation.

(7)
+static const char *
+pqGetProtocolMsgType(unsigned char c, bool toServer) {
+	if (toServer == true &&
+		c < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[c] != NULL)
+		return protocol_message_type_f[c];
+
+	if (toServer == false &&
+		c < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[c] != NULL)
+		return protocol_message_type_b[c];
+
+	return "UnknownMessage:";
+}

"== true/false" can be removed. libpq doesn't appear to use such style.

Yes, I removed it.

Why don't we embed this processing in pqTraceOutputMessage() because this
function is short and called only once? The added benefit would be that you
can print an invalid message type byte like this:

fprintf(..., "Unknown message: %02x\n", ...);

Sure. I fixed this message output.
And I removed switch-case default fprintf which output `Invalidate Protocol`.
Because Unknown...message covers this message meaning.

...

(8)
+ char id = '\0';

This initialization is not required because id will always be assigned a value
shortly.

I see, I removed this initialization.

(9)
+static int
+pqTraceOutputInt32(const char *data, FILE *pfdebug) {
+	int			result;
+
+	memcpy(&result, data, 4);
+	result = (int) pg_ntoh32(result);
+	fputc(' ', pfdebug);
+	fprintf(pfdebug, "%d", result);
+
+	return result;
+}

fputc() and fprintf() can be merged into one fprintf() call for efficiency.

The reason I separated this is because I thought it would be easy to handle any changes
that I wanted to make, such as changing spaces to tabs.
However this appear only 5 times. So I merged fprintf and fputc.

(10)
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f) {
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt32(message + cursor, f); }

I don't think you need to always use a cursor variable in order to align with
more complex messages. That is, you can just write:

+ pqTraceOutputInt32(message, f);
+ pqTraceOutputInt32(message + 4, f);

To follow Alvaro san's suggestion which is * number 4,
I left this cursor.

(11)
+		default:
+			fprintf(conn->Pfdebug, "Invalid Protocol:%c\n", id);
+			break;
+

(This is related to (7))
You can remove this default label if you exit the function before the switch
statement when the message type is unknown. Make sure to output \n in
that case.

I removed the default.

(12) pqTraceOutputB
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message + cursor, f);
+		cursor += 4;
+		if (nbytes == -1)
+			break;
+		pqTraceOutputNchar(message + cursor, f, nbytes);
+		cursor += nbytes;
+	}

Not break but continue, because non-NULL parameters may follow a NULL
parameter.

I changed break to continue.

(13) pqTraceOutputB
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 4;
+	pqTraceOutputInt16(message + cursor, f); }

This part doesn't seem to match the following description.

----------
After the last parameter, the following fields appear:

Int16
The number of result-column format codes that follow (denoted R below).
This can be zero to indicate that there are no result columns or that the result
columns should all use the default format (text); or one, in which case the
specified format code is applied to all result columns (if any); or it can equal
the actual number of result columns of the query.

Int16[R]
The result-column format codes. Each must presently be zero (text) or one
(binary).
---------

Thank you. I fixed it to use for loop.

(14)
The processing for CancelRequest message is missing?

This message does not have first 1 byte. So I remove it from my patch.
I create new function pqTraceOutputNoTypeByteMessage() and prepare it in the function.

(15)
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f) {
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message + cursor, f);
+	cursor++;
+
+	while (end > cursor)
+	{
+		pqTraceOutputInt16(message + cursor, f);
+		cursor += 2;
+	}
+}
+

According to the following description, for loop should be used.

----------
Int16
The number of columns in the data to be copied (denoted N below).

Int16[N]
The format codes to be used for each column. Each must presently be zero
(text) or one (binary). All must be zero if the overall copy format is textual.
----------

I fixed it.

(16)
(15) is also true for the processing of 'H' message.

Thank you. I fixed it.

(17) pqTraceOutputD
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message + cursor, f);
+			cursor += 4;
+			if (len == -1)
+				break;
+			pqTraceOutputNchar(message + cursor, f, len);
+			cursor += len;
+		}

Not break but continue, because non-NULL columns may follow a NULL
column.

I fixed it.

(18)
+		case 'E':	/* Execute(F) or Error Return(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd,
conn->Pfdebug, toServer);
+			break;

Error Return -> ErrorResponse

I fixed it.

(19) pqTraceOutputF
Check the protocol again. Two for loops should be required, one for format
codes and another for arguments.

Thank you. I fixed it.

(20)
+ if ( len != -1)

Remove extra space before len.

I remove this space.

(21)
I guess you intentionally omitted the following messages because they are
only used during connection establishment. I'm fine with it. I wrote this just
in case you missed them.

GSSENCRequest (F)
Int32(8)

GSSResponse (F)
Byte1('p')

PasswordMessage (F)
Byte1('p')

SASLInitialResponse (F)
Byte1('p')

SASLResponse (F)
Byte1('p')

Yes, I think it does not appear.

(22)
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f) {
+	int	cursor = 0;
+
+	pqTraceOutputInt16(message + cursor, f);
+	cursor += 2;
+	pqTraceOutputString(message + cursor, f);

Not Int16 but Int32.

I fixed it.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Wednesday, March 17, 2021 11:10 AM

* I would make the pqTraceOutputInt16() function and siblings advance
the cursor themselves, actually. I think something like this:
static int
pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
{
uint16 tmp;
int result;

memcpy(&tmp, data + *cursor, 2);
*cursor += 2;
result = (int) pg_ntoh16(tmp);
fprintf(pfdebug, " #%d", result);

return result;
}
(So the caller has to pass the original "data" pointer, not
"data+cursor"). This means the caller no longer has to do "cursor+=N"
at each place. Also, you get rid of the moveStrCursor() which does
not look good.

That makes sense, because in fact the patch mistakenly added 4 when it
should add 2. Also, the code would become smaller.

I changed 5 message data type function this style.

* I'm not fond of the idea of prefixing "#" for 16bit ints and no
prefix for 32bit ints. Seems obscure and the output looks weird.
I would use a one-letter prefix for each type, "w" for 32-bit ints
(stands for "word") and "h" for 16-bit ints (stands for "half-word").
Message length goes unadorned. Then you'd have lines like this

2021-03-15 08:10:44.324296 < RowDescription 35 h1 "spcoptions"
w1213 h5 w1009 h65535 w-1 h0
2021-03-15 08:10:44.324335 < DataRow 32 h1 w22
'{random_page_cost=3.0}'

Yes, actually I felt something similar. Taking a second thought, I think we
don't have to have any prefix because it doesn't help users. So we're
removing the prefix. We don't have any opinion on adding some prefix.

Having only # can be confusing for users. So I removed #.

* I don't like that pqTraceOutputS/H print nothing when !toServer. I
think they should complain.

Yes, the caller should not call the function if there's no message content.
That way, the function doesn't need the toServer argument.

I moved pqTraceOutputS/H's `if(!toServer)` check to pqTraceOutputMessage() switch statement.

Regards,
Aya Iwata

Attachments:

v26-libpq-trace-log.patchapplication/octet-stream; name=v26-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index be674fb..3a5456d 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6459,12 +6459,27 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message type, message length, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      For numerical quantities, 16-bit values are preceded with <literal>#</literal>
+      and 32-bit values are printed without a prefix.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -6479,6 +6494,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..0424523 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5c48c14..a00701f 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -183,3 +183,4 @@ PQenterPipelineMode       180
 PQexitPipelineMode        181
 PQpipelineSync            182
 PQpipelineStatus          183
+PQtraceSetFlags           184
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 53b354a..a90bdb8 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6859,27 +6859,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index f344370..19e8a4b 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -970,10 +970,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..732fefb 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -84,9 +84,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +97,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +132,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +157,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +175,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +192,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +207,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +244,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +277,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +485,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +519,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +528,16 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug && conn->outCount < conn->outMsgStart)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +964,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 306e89a..de77c06 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -457,6 +457,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1660,6 +1663,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2121,6 +2127,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index cee42d4..d0decde 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -376,7 +376,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6374ec6..1a1d0e1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -818,6 +819,12 @@ extern ssize_t pg_GSS_write(PGconn *conn, const void *ptr, size_t len);
 extern ssize_t pg_GSS_read(PGconn *conn, void *ptr, size_t len);
 #endif
 
+/* === in libpq-trace.c === */
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message,
+															bool toServer);
+extern void pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message);
+
 /* === miscellaneous macros === */
 
 /*
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..2c88ef7
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,769 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+{
+	const char *v = data + *cursor;
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*v))
+		fprintf(pfdebug, " \\x%02x", *v);
+	else
+		fprintf(pfdebug, " %c", *v);
+	++*cursor;
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+{
+	uint16		tmp;
+	int			result;
+
+	memcpy(&tmp, data + *cursor , 2);
+	*cursor += 2;
+	result = (int) pg_ntoh16(tmp);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data + *cursor, 4);
+	*cursor += 4;
+	result = (int) pg_ntoh32(result);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(const char *data, int *cursor, int end, FILE *pfdebug)
+{
+	fprintf(pfdebug, " \"%s\"", data + *cursor);
+
+	while (*cursor < end && data[*cursor])
+		++*cursor;
+	++*cursor;
+}
+
+/*
+ * pqTraceOutputBinary: output a string possibly consisting of
+ * non-printable characters. Hex representation is used for such
+ * chars; others are printed normally.
+ *
+ * XXX this probably doesn't do a great job with multibyte chars, but then we
+ * don't know what is text and what encoding it'd be in.
+ */
+static void
+pqTraceOutputBinary(const char *v, int length, FILE *pfdebug)
+{
+	int			i,
+				next;			/* first char not yet printed */
+
+	for (next = i = 0; i < length; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < length)
+		fwrite(v + next, 1, length - next, pfdebug);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+{
+	fprintf(pfdebug, " \'");
+	pqTraceOutputBinary(data + *cursor, len, pfdebug);
+	fprintf(pfdebug, "\'");
+	*cursor += len;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(const char *message, int end, FILE *f)
+{
+	int cursor = 0;
+	int nparams;
+	int nbytes;
+	int i;
+
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, end, f);
+	}
+	else
+		pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message, &cursor, f);
+
+	while (end > cursor)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, end, f);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		nfields = pqTraceOutputInt16(message, &cursor, f);
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message, &cursor, f);
+			if (len == -1)
+				continue;
+			pqTraceOutputNchar(message, &cursor, f, len);
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+	}
+	else
+	{
+		while (end > cursor)
+		{
+			pqTraceOutputByte1(message, &cursor, f);
+			if (message[cursor] == '\0')
+				continue;
+			pqTraceOutputString(message, &cursor, end, f);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int nbytes;
+	int	i;
+
+	pqTraceOutputInt32(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int		len;
+
+	len = pqTraceOutputInt32(message, &cursor, f);
+	if (len != -1)
+		pqTraceOutputNchar(message, &cursor, f, len);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	while (end > cursor)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		if (message[cursor] == '\0')
+			continue;
+		pqTraceOutputString(message, &cursor, end, f);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* Parse */
+static void
+pqTraceOutputP(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nparams;
+	int i;
+
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Query */
+static void
+pqTraceOutputQ(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputByte1(message, &cursor, f);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	char		timestr[128];
+	char 		id;
+	int			length;
+	char	   *prefix = toServer ? ">" : "<";
+	int			LogCursor = 0;
+	int			LogEnd;
+	const char	*message_type = "UnkownMessage";
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+
+	/* Get a protocol type from first byte identifier */
+	if (toServer &&
+		id < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_f[(unsigned char)id];
+	else if (!toServer &&
+		id < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_b[(unsigned char)id];
+	else
+	{
+		fprintf(conn->Pfdebug, "Unknown message: %02x\n", id);
+		return;
+	}
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%s\t%d", timestr, prefix, message_type, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (!toServer)
+				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'E':	/* Execute(F) or Error Response(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (!toServer)
+				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case '2':	/* Bind Complete */
+		case '3':	/* Close Complete */
+		case 'c':	/* Copy Done */
+		case 'I':	/* empty query */
+		case 'n':	/* No Data */
+		case '1':	/* Parse Complete */
+		case 's':	/* Portal Suspended */
+		case 'X':	/* Terminate */
+			/* No message content */
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
+
+void
+pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
+{
+	char		timestr[128];
+	int			length;
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	switch(length)
+	{
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "%s\t>\tCancelRequest\t%d", timestr, length);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			break;
+		case 8 :	/* GSSENRequest or SSLRequest */
+			/* These messsage does not reach here. */
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: length is %d", length);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
#170Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#169)
Re: libpq debug log

Hello

In pqTraceOutputString(), you can use the return value from fprintf to
move the cursor -- no need to count chars.

I still think that the message-type specific functions should print the
message type, rather than having the string arrays.

--
�lvaro Herrera Valdivia, Chile
"La gente vulgar s�lo piensa en pasar el tiempo;
el que tiene talento, en aprovecharlo"

#171tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#169)
RE: libpq debug log

I'll look at the comments from Alvaro-san and Horiguchi-san. Here are my review comments:

(23)
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug && conn->outCount < conn->outMsgStart)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}

The inner else path doesn't seem to be reached because both the outer and inner if contain the same condition. I think you want to remove the second condition from the outer if.

(24) pqTraceOutputNoTypeByteMessage
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "%s\t>\tCancelRequest\t%d", timestr, length);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			break;

Another int32 data needs to be output as follows:

--------------------------------------------------
Int32(80877102)
The cancel request code. The value is chosen to contain 1234 in the most significant 16 bits, and 5678 in the least significant 16 bits. (To avoid confusion, this code must not be the same as any protocol version number.)

Int32
The process ID of the target backend.

Int32
The secret key for the target backend.
--------------------------------------------------

(25)
+ case 8 : /* GSSENRequest or SSLRequest */

GSSENRequest -> GSSENCRequest

(26)
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+{
+	const char *v = data + *cursor;
+	/*
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputString(message, &cursor, end, f);
+}

Put an empty line to separate the declaration part and execution part.

(27)
+	const char	*message_type = "UnkownMessage";
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+	/* Get a protocol type from first byte identifier */
+	if (toServer &&
+		id < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_f[(unsigned char)id];
+	else if (!toServer &&
+		id < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_b[(unsigned char)id];
+	else
+	{
+		fprintf(conn->Pfdebug, "Unknown message: %02x\n", id);
+		return;
+	}
+

The initial value "UnkownMessage" is not used. So, you can initialize message_type with NULL and do like:

+	 if (...)
+		...
+	else if (...)
+		...
+
+	if (message_type == NULL)
+	{
+		fprintf(conn->Pfdebug, "Unknown message: %02x\n", id);
+		return;
+

Plus, I think this should be done before looking at the message length.

(28)
pqTraceOutputBinary() is only used in pqTraceOutputNchar(). Do they have to be separated?

Regards
Takayuki Tsunakawa

#172tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Alvaro Herrera (#170)
RE: libpq debug log

From: Alvaro Herrera <alvherre@alvh.no-ip.org>

In pqTraceOutputString(), you can use the return value from fprintf to
move the cursor -- no need to count chars.

Yes, precisely, 2 bytes for the double quotes needs to be subtracted as follows:

len = fprintf(...);
*cursor += (len - 2);

I still think that the message-type specific functions should print the
message type, rather than having the string arrays.

I sort of think so to remove the big arrays. But to minimize duplicate code, I think the code structure will look like:

fprintf(timestamp, length);
switch (type)
{
case '?':
pqTraceOutput?(...);
break;
case '?':
/* No message content */
fprintf("message_type_name");
break;
}

void
pqTraceOutput?(...)
{
fprintf("message_type_name");
print message content;
}

The order of length and message type is reversed. The .sgml should also be changed accordingly. What do you think?

Iwata-san,
Why don't you submit v27 patch with the current arrays kept, and then v28 with the arrays removed soon after?

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

It would help when the value is "255", which is confusing between -1
(or 255) in byte or 255 in 2-byte word. Or when the value looks like
broken. I'd suggest "b"yte for 1 byte, "s"hort for 2 bytes, "l"ong for
4 bytes ('l' is confusing with '1', but anyway it is not needed)..

I don't have a strong opinion on this. (I kind of think I want to see unprefixed numbers; readers will look at the protocol reference anyway.) I'd like to leave this up to Iwata-san and Alvaro-san.

Regards
Takayuki Tsunakawa}

#173iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#171)
1 attachment(s)
RE: libpq debug log

Hi Alvaro san and Tsunakawa san,

Thank you for your review. I updated patch to v27.

`make check` output is following. I think it is OK.
```
2021-03-18 07:02:55.090598 < ReadyForQuery 5 I
2021-03-18 07:02:55.090672 > Terminate 4
2021-03-18 07:02:55.120492 > Query 155 "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (some_nonexistent_parameter = true);"
2021-03-18 07:02:55.121624 < ErrorResponse 124 S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L "1447" R "parseRelOptionsInternal" \x00 "Z"
2021-03-18 07:02:55.121664 < ReadyForQuery 5 I
2021-03-18 07:02:55.121728 > Query 144 "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (random_page_cost = 3.0);"
2021-03-18 07:02:55.123335 < CommandComplete 22 "CREATE TABLESPACE"
2021-03-18 07:02:55.123400 < ReadyForQuery 5 I
2021-03-18 07:02:55.123460 > Query 81 "SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';"
2021-03-18 07:02:55.126556 < RowDescription 35 1 "spcoptions" 1213 5 1009 65535 -1 0
2021-03-18 07:02:55.126594 < DataRow 32 1 22 '{random_page_cost=3.0}'
2021-03-18 07:02:55.126607 < CommandComplete 13 "SELECT 1"
2021-03-18 07:02:55.126617 < ReadyForQuery 5 I
```

Iwata-san,
Why don't you submit v27 patch with the current arrays kept, and then v28
with the arrays removed soon after?

And I will try to remove byte1 type table for v28 patch.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Thursday, March 18, 2021 12:38 PM
From: Alvaro Herrera <alvherre@alvh.no-ip.org>

In pqTraceOutputString(), you can use the return value from fprintf to
move the cursor -- no need to count chars.

Yes, precisely, 2 bytes for the double quotes needs to be subtracted as
follows:

len = fprintf(...);
*cursor += (len - 2);

Thank you for your advice. I changed pqTraceOutputString set cursor to fprintf return -2.
And I removed cursor movement from that function.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Thursday, March 18, 2021 11:52 AM

...

I'll look at the comments from Alvaro-san and Horiguchi-san. Here are my
review comments:

Thank you for your review. I am sorry previous patch contain some mistake.

(23)
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug && conn->outCount < conn->outMsgStart)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer +
conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+
conn->outBuffer + conn->outMsgStart);
+	}

The inner else path doesn't seem to be reached because both the outer and
inner if contain the same condition. I think you want to remove the second
condition from the outer if.

Yes, I remove second condition.

(24) pqTraceOutputNoTypeByteMessage
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug,
"%s\t>\tCancelRequest\t%d", timestr, length);
+			pqTraceOutputInt32(message, &LogCursor,
conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor,
conn->Pfdebug);
+			break;

Another int32 data needs to be output as follows:

--------------------------------------------------
Int32(80877102)
The cancel request code. The value is chosen to contain 1234 in the most
significant 16 bits, and 5678 in the least significant 16 bits. (To avoid
confusion, this code must not be the same as any protocol version number.)

Int32
The process ID of the target backend.

Int32
The secret key for the target backend.
--------------------------------------------------

Thank you. I read document again and I add one pqTraceOutputInt32().

(25)
+ case 8 : /* GSSENRequest or SSLRequest */

GSSENRequest -> GSSENCRequest

Thank you. I fixed.

(26)
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug) {
+	const char *v = data + *cursor;
+	/*
+static void
+pqTraceOutputf(const char *message, int end, FILE *f) {
+	int	cursor = 0;
+	pqTraceOutputString(message, &cursor, end, f); }

Put an empty line to separate the declaration part and execution part.

Thank you. I fixed this. I add space anywhere in pqTraceOutput? function.

(27)
+	const char	*message_type = "UnkownMessage";
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+	/* Get a protocol type from first byte identifier */
+	if (toServer &&
+		id < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_f[(unsigned
char)id];
+	else if (!toServer &&
+		id < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_b[(unsigned
char)id];
+	else
+	{
+		fprintf(conn->Pfdebug, "Unknown message: %02x\n", id);
+		return;
+	}
+

The initial value "UnkownMessage" is not used. So, you can initialize
message_type with NULL and do like:

+	 if (...)
+		...
+	else if (...)
+		...
+
+	if (message_type == NULL)
+	{
+		fprintf(conn->Pfdebug, "Unknown message: %02x\n", id);
+		return;
+

Plus, I think this should be done before looking at the message length.

I initialized message_type as NULL, changed `else` to `if (message_type == NULL)`
and move message type setup code to after setting id.

(28)
pqTraceOutputBinary() is only used in pqTraceOutputNchar(). Do they have
to be separated?

I see. I merged this code to pqTraceOutputBinary().

Regards,
Aya Iwata

Attachments:

v27-libpq-trace-log.patchapplication/octet-stream; name=v27-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index be674fb..3a5456d 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6459,12 +6459,27 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message type, message length, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      For numerical quantities, 16-bit values are preceded with <literal>#</literal>
+      and 32-bit values are printed without a prefix.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -6479,6 +6494,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..0424523 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5c48c14..a00701f 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -183,3 +183,4 @@ PQenterPipelineMode       180
 PQexitPipelineMode        181
 PQpipelineSync            182
 PQpipelineStatus          183
+PQtraceSetFlags           184
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 53b354a..a90bdb8 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6859,27 +6859,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index f344370..19e8a4b 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -970,10 +970,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..54d6a76 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -84,9 +84,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +97,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +132,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +157,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +175,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +192,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +207,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +244,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +277,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +485,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +519,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +528,16 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +964,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 306e89a..de77c06 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -457,6 +457,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1660,6 +1663,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2121,6 +2127,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index cee42d4..d0decde 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -376,7 +376,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6374ec6..1a1d0e1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -818,6 +819,12 @@ extern ssize_t pg_GSS_write(PGconn *conn, const void *ptr, size_t len);
 extern ssize_t pg_GSS_read(PGconn *conn, void *ptr, size_t len);
 #endif
 
+/* === in libpq-trace.c === */
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message,
+															bool toServer);
+extern void pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message);
+
 /* === miscellaneous macros === */
 
 /*
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..0863650
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,764 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+/*
+ * protocol message types:
+ *
+ * protocol_message_type_b[]: message types sent by a backend
+ * protocol_message_type_f[]: message types sent by a frontend
+ */
+static const char *const protocol_message_type_b[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0,							/* 0 */
+	"ParseComplete",			/* 1 */
+	"BindComplete",				/* 2 */
+	"CloseComplete",			/* 3 */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x34 ... \x3f */
+	0,							/* @ */
+	"NotificationResponse",		/* A */
+	0,							/* B */
+	"CommandComplete",			/* C */
+	"DataRow",					/* D */
+	"ErrorResponse",			/* E */
+	0,							/* F */
+	"CopyInResponse",			/* G */
+	"CopyOutResponse",			/* H */
+	"EmptyQueryResponse",		/* I */
+	0,							/* J */
+	"BackendKeyData",			/* K */
+	0,							/* L */
+	0,							/* M */
+	"NoticeResponse",			/* N */
+	0,							/* O */
+	0,							/* P */
+	0,							/* Q */
+	"Authentication",			/* R */
+	"ParameterStatus",			/* S */
+	"RowDescription",			/* T */
+	0,							/* U */
+	"FunctionCallResponse",		/* V */
+	"CopyBothResponse",			/* W */
+	0,							/* X */
+	0,							/* Y */
+	"ReadyForQuery",			/* Z */
+	0, 0, 0, 0, 0,				/* \x5b ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* \x65 ... \0x6d */
+	"NoData",					/* n */
+	0,							/* o */
+	0,							/* p */
+	0,							/* q */
+	0,							/* r */
+	"PortalSuspended",			/* s */
+	"ParameterDescription",		/* t */
+	0,							/* u */
+	"NegotiateProtocolVersion", /* v */
+};
+
+static const char *const protocol_message_type_f[] = {
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x00 ... \x0f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x10 ... \x1f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x20 ... \x2f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \x30 ... \x3f */
+	0,							/* @ */
+	0,							/* A */
+	"Bind",						/* B */
+	"Close",					/* C */
+	"Describe",					/* D */
+	"Execute",					/* E */
+	"FunctionCall",				/* F */
+	0,							/* G */
+	"Flush",					/* H */
+	0, 0, 0, 0, 0, 0, 0,		/* I ... O */
+	"Parse",					/* P */
+	"Query",					/* Q */
+	0,							/* R */
+	"Sync",						/* S */
+	0, 0, 0, 0,					/* T ... W */
+	"Terminate",				/* X */
+	0, 0, 0, 0, 0, 0, 0,		/* \x59 ... \x5f */
+	0,							/* ` */
+	0,							/* a */
+	0,							/* b */
+	"CopyDone",					/* c */
+	"CopyData",					/* d */
+	0,							/* e */
+	"CopyFail",					/* f */
+	0, 0, 0, 0, 0, 0, 0, 0, 0,	/* g ... o */
+	"AuthenticationResponse",	/* p */
+};
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+{
+	const char *v = data + *cursor;
+
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*v))
+		fprintf(pfdebug, " \\x%02x", *v);
+	else
+		fprintf(pfdebug, " %c", *v);
+	++*cursor;
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+{
+	uint16		tmp;
+	int			result;
+
+	memcpy(&tmp, data + *cursor , 2);
+	*cursor += 2;
+	result = (int) pg_ntoh16(tmp);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data + *cursor, 4);
+	*cursor += 4;
+	result = (int) pg_ntoh32(result);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(const char *data, int *cursor, int end, FILE *pfdebug)
+{
+	int	len;
+
+	len = fprintf(pfdebug, " \"%s\"", data + *cursor);
+	*cursor += (len - 2);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+{
+	int			i,
+				next;			/* first char not yet printed */
+	const char	*v = data + *cursor;
+
+	fprintf(pfdebug, " \'");
+
+	for (next = i = 0; i < len; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < len)
+		fwrite(v + next, 1, len - next, pfdebug);
+
+	fprintf(pfdebug, "\'");
+	*cursor += len;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(const char *message, int end, FILE *f)
+{
+	int cursor = 0;
+	int nparams;
+	int nbytes;
+	int i;
+
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, end, f);
+	}
+	else
+		pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message, &cursor, f);
+
+	while (end > cursor)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, end, f);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		nfields = pqTraceOutputInt16(message, &cursor, f);
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message, &cursor, f);
+			if (len == -1)
+				continue;
+			pqTraceOutputNchar(message, &cursor, f, len);
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+	}
+	else
+	{
+		while (end > cursor)
+		{
+			pqTraceOutputByte1(message, &cursor, f);
+			if (message[cursor] == '\0')
+				continue;
+			pqTraceOutputString(message, &cursor, end, f);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int nbytes;
+	int	i;
+
+	pqTraceOutputInt32(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int		len;
+
+	len = pqTraceOutputInt32(message, &cursor, f);
+	if (len != -1)
+		pqTraceOutputNchar(message, &cursor, f, len);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	while (end > cursor)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		if (message[cursor] == '\0')
+			continue;
+		pqTraceOutputString(message, &cursor, end, f);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* Parse */
+static void
+pqTraceOutputP(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nparams;
+	int i;
+
+	pqTraceOutputString(message, &cursor, end, f);
+	pqTraceOutputString(message, &cursor, end, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Query */
+static void
+pqTraceOutputQ(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputString(message, &cursor, end, f);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	pqTraceOutputByte1(message, &cursor, f);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	char		timestr[128];
+	char 		id;
+	int			length;
+	char	   *prefix = toServer ? ">" : "<";
+	int			LogCursor = 0;
+	int			LogEnd;
+	const char	*message_type = NULL;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	id = message[LogCursor++];
+
+	/* Get a protocol type from first byte identifier */
+	if (toServer &&
+		id < lengthof(protocol_message_type_f) &&
+		protocol_message_type_f[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_f[(unsigned char)id];
+	else if (!toServer &&
+		id < lengthof(protocol_message_type_b) &&
+		protocol_message_type_b[(unsigned char)id] != NULL)
+		message_type = protocol_message_type_b[(unsigned char)id];
+
+	if (message_type == NULL)
+	{
+		fprintf(conn->Pfdebug, "Unknown message: %02x\n", id);
+		return;
+	}
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%s\t%d", timestr, prefix, message_type, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (!toServer)
+				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'E':	/* Execute(F) or Error Response(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (!toServer)
+				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case '2':	/* Bind Complete */
+		case '3':	/* Close Complete */
+		case 'c':	/* Copy Done */
+		case 'I':	/* empty query */
+		case 'n':	/* No Data */
+		case '1':	/* Parse Complete */
+		case 's':	/* Portal Suspended */
+		case 'X':	/* Terminate */
+			/* No message content */
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
+
+void
+pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
+{
+	char		timestr[128];
+	int			length;
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	switch(length)
+	{
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "%s\t>\tCancelRequest\t%d", timestr, length);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			break;
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messsage does not reach here. */
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: length is %d", length);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
#174tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#173)
RE: libpq debug log

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

Yes, precisely, 2 bytes for the double quotes needs to be subtracted
as
follows:

len = fprintf(...);
*cursor += (len - 2);

Thank you for your advice. I changed pqTraceOutputString set cursor to fprintf
return -2.
And I removed cursor movement from that function.

Ouch, not 2 but 3, to include a single whitespace at the beginning.

The rest looks good. I hope we're almost at the finish line.

Regards
Takayuki Tsunakawa}

#175Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: tsunakawa.takay@fujitsu.com (#171)
Re: libpq debug log

At Thu, 18 Mar 2021 07:34:36 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

Yes, precisely, 2 bytes for the double quotes needs to be subtracted
as
follows:

len = fprintf(...);
*cursor += (len - 2);

Thank you for your advice. I changed pqTraceOutputString set cursor to fprintf
return -2.
And I removed cursor movement from that function.

Ouch, not 2 but 3, to include a single whitespace at the beginning.

The rest looks good. I hope we're almost at the finish line.

Maybe.

At Wed, 17 Mar 2021 13:36:32 -0300, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in

In pqTraceOutputString(), you can use the return value from fprintf to
move the cursor -- no need to count chars.

I still think that the message-type specific functions should print the
message type, rather than having the string arrays.

In other words, pqTraceOutputMessage recognizes message id and calls
the function corresponding one-on-one to the id. So the functions
knows what is the message type of myself and there's no reason for
pqTraceOutputMessage to print the message type on its behalf.

+ pqTraceOutputR(const char *message, FILE *f)
+ {
+ 	int	cursor = 0;
+ 
+ 	pqTraceOutputInt32(message, &cursor, f);

I don't understand the reason for spliting message and &cursor here.

+ pqTraceOutputR(const char *message, FILE *f)
+ {
+ 	char *p = message;
+ 
+ 	pqTraceOutputInt32(&p, f);

works well.

+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}

I didn't looked closer, but lookong the usage of the variable "end",
something's wrong in the function.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#176tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#175)
RE: libpq debug log

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

+ pqTraceOutputR(const char *message, FILE *f)
+ {
+ 	int	cursor = 0;
+
+ 	pqTraceOutputInt32(message, &cursor, f);

I don't understand the reason for spliting message and &cursor here.

+ pqTraceOutputR(const char *message, FILE *f)
+ {
+ 	char *p = message;
+
+ 	pqTraceOutputInt32(&p, f);

works well.

Yes, that would also work. But I like the separate cursor + fixed starting point here, probably because it's sometimes confusing to see the pointer value changed inside functions. (And a pointer itself is an allergy for some people, not to mention a pointer to ointer...) Also, libpq uses cursors for network I/O buffers. So, I think the patch can be as it is now.

+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}

I didn't looked closer, but lookong the usage of the variable "end",
something's wrong in the function.

Ah, end doesn't serve any purpose. I guess the compiler emitted a warning "end is not used". I think end can be removed.

Regards
Takayuki Tsunakawa}

#177iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: Kyotaro Horiguchi (#175)
1 attachment(s)
RE: libpq debug log

Hi Horiguchi san and Tsunakawa san,

Thank you for you review.

I update patch to v28. In this patch, I removed array.
And I fixed some code according to Horiguchi san and Tsunakawa san review comment.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Thursday, March 18, 2021 12:38 PM
I sort of think so to remove the big arrays. But to minimize duplicate code, I
think the code structure will look like:

fprintf(timestamp, length);
switch (type)
{
case '?':
pqTraceOutput?(...);
break;
case '?':
/* No message content */
fprintf("message_type_name");
break;
}

void
pqTraceOutput?(...)
{
fprintf("message_type_name");
print message content;
}

The code follows above format.
And I changed .sgml documentation;
- Changed order of message length and type
- Removed byte-16 and byte-32 explanation because I removed # output in previous patch.

Output style is following;

```
2021-03-18 08:59:36.141660 < 5 ReadyForQuery I
2021-03-18 08:59:36.141723 > 4 Terminate
2021-03-18 08:59:36.173263 > 155 Query "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (some_nonexistent_parameter = true);"
2021-03-18 08:59:36.174439 < 124 ErrorResponse S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L "1456" R "parseRelOptionsInternal" \x00 "Z"
2021-03-18 08:59:36.174483 < 5 ReadyForQuery I
2021-03-18 08:59:36.174545 > 144 Query "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (random_page_cost = 3.0);"
2021-03-18 08:59:36.176155 < 22 CommandComplete "CREATE TABLESPACE"
2021-03-18 08:59:36.176190 < 5 ReadyForQuery I
2021-03-18 08:59:36.176243 > 81 Query "SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';"
2021-03-18 08:59:36.179286 < 35 RowDescription 1 "spcoptions" 1213 5 1009 65535 -1 0
2021-03-18 08:59:36.179326 < 32 DataRow 1 22 '{random_page_cost=3.0}'
2021-03-18 08:59:36.179339 < 13 CommandComplete "SELECT 1"
2021-03-18 08:59:36.179349 < 5 ReadyForQuery I
2021-03-18 08:59:36.179504 > 42 Query "DROP TABLESPACE regress_tblspacewith;"
2021-03-18 08:59:36.180400 < 20 CommandComplete "DROP TABLESPACE"
2021-03-18 08:59:36.180432 < 5 ReadyForQuery I
```

-----Original Message-----
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Thursday, March 18, 2021 5:30 PM

At Thu, 18 Mar 2021 07:34:36 +0000, "tsunakawa.takay@fujitsu.com"
<tsunakawa.takay@fujitsu.com> wrote in

From: Iwata, Aya/岩田 彩 <iwata.aya@fujitsu.com>

Yes, precisely, 2 bytes for the double quotes needs to be
subtracted as
follows:

len = fprintf(...);
*cursor += (len - 2);

Thank you for your advice. I changed pqTraceOutputString set cursor
to fprintf return -2.
And I removed cursor movement from that function.

Ouch, not 2 but 3, to include a single whitespace at the beginning.

The rest looks good. I hope we're almost at the finish line.

Maybe.

String is end by '\0'. So (len -2) is OK.
However it seems like mistake because fprintf output string and 3 characters.
So I added explanation here and changed (len -2) to (len -3 +1).
I think it is OK because I found similar style in ecpg code.

+ pqTraceOutputR(const char *message, FILE *f) {
+ 	int	cursor = 0;
+
+ 	pqTraceOutputInt32(message, &cursor, f);

I don't understand the reason for spliting message and &cursor here.

+ pqTraceOutputR(const char *message, FILE *f) {
+ 	char *p = message;
+
+ 	pqTraceOutputInt32(&p, f);

works well.

Yes, that would also work. But I like the separate cursor + fixed starting
point here, probably because it's sometimes confusing to see the pointer
value changed inside functions. (And a pointer itself is an allergy for some
people, not to mention a pointer to ointer...) Also, libpq uses cursors for
network I/O buffers. So, I think the patch can be as it is now.

I think pass message and cursor is better. Because it is easy to understand and
The moving cursor style is used when libpq execute protocol message.
So I didn't make this change.

+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f) {
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, end, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}

I didn't looked closer, but lookong the usage of the variable "end", something's
wrong in the function.

I removed end from the function.
pqTraceOutputString no longer use message end cursor.

Regards,
Aya Iwata

Attachments:

v28-libpq-trace-log.patchapplication/octet-stream; name=v28-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index be674fb..00287cb 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6459,12 +6459,25 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),
+      message length, message type, and message contents.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -6479,6 +6492,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..0424523 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5c48c14..a00701f 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -183,3 +183,4 @@ PQenterPipelineMode       180
 PQexitPipelineMode        181
 PQpipelineSync            182
 PQpipelineStatus          183
+PQtraceSetFlags           184
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 53b354a..a90bdb8 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6859,27 +6859,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index f344370..19e8a4b 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -970,10 +970,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..54d6a76 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -84,9 +84,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +97,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +132,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +157,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +175,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +192,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +207,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +244,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +277,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +485,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +519,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +528,16 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +964,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 306e89a..de77c06 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -457,6 +457,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1660,6 +1663,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2121,6 +2127,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index cee42d4..d0decde 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -376,7 +376,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6374ec6..1a1d0e1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -818,6 +819,12 @@ extern ssize_t pg_GSS_write(PGconn *conn, const void *ptr, size_t len);
 extern ssize_t pg_GSS_read(PGconn *conn, void *ptr, size_t len);
 #endif
 
+/* === in libpq-trace.c === */
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message,
+															bool toServer);
+extern void pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message);
+
 /* === miscellaneous macros === */
 
 /*
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..13853b1
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,712 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+{
+	const char *v = data + *cursor;
+
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*v))
+		fprintf(pfdebug, " \\x%02x", *v);
+	else
+		fprintf(pfdebug, " %c", *v);
+	++*cursor;
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+{
+	uint16		tmp;
+	int			result;
+
+	memcpy(&tmp, data + *cursor , 2);
+	*cursor += 2;
+	result = (int) pg_ntoh16(tmp);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data + *cursor, 4);
+	*cursor += 4;
+	result = (int) pg_ntoh32(result);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
+{
+	int	len;
+
+	len = fprintf(pfdebug, " \"%s\"", data + *cursor);
+
+	/*
+	 * This is null-terminated string. So add 1 after subtracting 3
+	 * which is the double quotes and space length from len.
+	 */
+	*cursor += (len - 3 + 1);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+{
+	int			i,
+				next;			/* first char not yet printed */
+	const char	*v = data + *cursor;
+
+	fprintf(pfdebug, " \'");
+
+	for (next = i = 0; i < len; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < len)
+		fwrite(v + next, 1, len - next, pfdebug);
+
+	fprintf(pfdebug, "\'");
+	*cursor += len;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tAuthentication");
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tBackendKeyData");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(const char *message, int end, FILE *f)
+{
+	int cursor = 0;
+	int nparams;
+	int nbytes;
+	int i;
+
+	fprintf(f, "\tBind");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "\tClose");
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, f);
+	}
+	else
+	{
+		fprintf(f, "\tCommandComplete");
+		pqTraceOutputString(message, &cursor, f);
+	}
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\nCopyFail");
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "\tCopyInResponse");
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "\tCopyOutResponse");
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tCopyBothResponse");
+	pqTraceOutputByte1(message, &cursor, f);
+
+	while (end > cursor)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "\tDescribe");
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, f);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		fprintf(f, "\tDataRow");
+		nfields = pqTraceOutputInt16(message, &cursor, f);
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message, &cursor, f);
+			if (len == -1)
+				continue;
+			pqTraceOutputNchar(message, &cursor, f, len);
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "\tExecute");
+		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+	}
+	else
+	{
+		fprintf(f, "\tErrorResponse");
+		while (end > cursor)
+		{
+			pqTraceOutputByte1(message, &cursor, f);
+			if (message[cursor] == '\0')
+				continue;
+			pqTraceOutputString(message, &cursor, f);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int nbytes;
+	int	i;
+
+	fprintf(f, "\tFunctionCall");
+	pqTraceOutputInt32(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int		len;
+
+	fprintf(f, "\tFunctionCallResponse");
+	len = pqTraceOutputInt32(message, &cursor, f);
+	if (len != -1)
+		pqTraceOutputNchar(message, &cursor, f, len);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tNegotiateProtocolVersion");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tNoticeResponse");
+	while (end > cursor)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		if (message[cursor] == '\0')
+			continue;
+		pqTraceOutputString(message, &cursor, f);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tNotificationResponse");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "\tParameterDescription");
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tParameterStatus");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* Parse */
+static void
+pqTraceOutputP(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nparams;
+	int i;
+
+	fprintf(f, "\tParse");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Query */
+static void
+pqTraceOutputQ(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tQuery");
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "\tReadyForQuery");
+	pqTraceOutputByte1(message, &cursor, f);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	fprintf(f, "\tRowDescription");
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	char		timestr[128];
+	char 		id;
+	int			length;
+	char	   *prefix = toServer ? ">" : "<";
+	int			LogCursor = 0;
+	int			LogEnd;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%d", timestr, prefix, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (!toServer)
+				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+			else
+				fprintf(conn->Pfdebug, "\tFlush");
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'E':	/* Execute(F) or Error Response(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (!toServer)
+				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+			else
+				fprintf(conn->Pfdebug, "\tSync");
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case '2':	/* Bind Complete */
+			fprintf(conn->Pfdebug, "\tBindComplete");
+			/* No message content */
+			break;
+		case '3':	/* Close Complete */
+			fprintf(conn->Pfdebug, "\tCloseComplete");
+			/* No message content */
+			break;
+		case 'c':	/* Copy Done */
+			fprintf(conn->Pfdebug, "\tCopyDone");
+			/* No message content */
+			break;
+		case 'I':	/* Empty Query Response */
+			fprintf(conn->Pfdebug, "\tEmptyQueryResponse");
+			/* No message content */
+			break;
+		case 'n':	/* No Data */
+			fprintf(conn->Pfdebug, "\tNoData");
+			/* No message content */
+			break;
+		case '1':	/* Parse Complete */
+			fprintf(conn->Pfdebug, "\tParseComplete");
+			/* No message content */
+			break;
+		case 's':	/* Portal Suspended */
+			fprintf(conn->Pfdebug, "\tPortalSuspended");
+			/* No message content */
+			break;
+		case 'X':	/* Terminate */
+			fprintf(conn->Pfdebug, "\tTerminate");
+			/* No message content */
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
+
+void
+pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
+{
+	char		timestr[128];
+	int			length;
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	switch(length)
+	{
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "%s\t>\t%d\tCancelRequest", timestr, length);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			break;
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messsage does not reach here. */
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: length is %d", length);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
#178tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#177)
RE: libpq debug log

Hello Iwata-san,

Thanks to removing the static arrays, the code got much smaller. I'm happy with this result.

(1)
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),

I think this was meant to say "> or <". So, maybe you should remove "," at the end of the first line, and replace "and" with "or".

(2)
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messsage does not reach here. */

messsage does -> messages do

(3)
+ fprintf(f, "\tAuthentication");

Why don't you move this \t in each message type to the end of:

+ fprintf(conn->Pfdebug, "%s\t%s\t%d", timestr, prefix, length);

Plus, don't we say in the manual that fields (timestamp, direction, length, message type, and message content) are separated by a tab?
Also, the code doesn't seem to separate the message type and content with a tab.

(4)
Where did the processing for unknown message go in pqTraceOutputMessage()? Add default label?

(5)
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "%s\t>\t%d\tCancelRequest", timestr, length);

I think this should follow pqTraceOutputMessage(). That is, each case label only print the message type name in case other message types are added in the future.

Regards
Takayuki Tsunakawa

#179iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#178)
1 attachment(s)
RE: libpq debug log

Hi Tsunakawa san,

Thank you for your review. I update patch to v29.

Output is following. It is fine.
```
2021-03-19 07:21:09.917302 > 4 Terminate
2021-03-19 07:21:09.961494 > 155 Query "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (some_nonexistent_parameter = true);"
2021-03-19 07:21:09.962657 < 124 ErrorResponse S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L "1456" R "parseRelOptionsInternal" \x00 "Z"
2021-03-19 07:21:09.962702 < 5 ReadyForQuery I
2021-03-19 07:21:09.962767 > 144 Query "CREATE TABLESPACE regress_tblspacewith LOCATION '/home/iwata/pgsql/postgres/src/test/regress/testtablespace' WITH (random_page_cost = 3.0);"
2021-03-19 07:21:09.967056 < 22 CommandComplete "CREATE TABLESPACE"
2021-03-19 07:21:09.967092 < 5 ReadyForQuery I
2021-03-19 07:21:09.967148 > 81 Query "SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';"
2021-03-19 07:21:09.970338 < 35 RowDescription 1 "spcoptions" 1213 5 1009 65535 -1 0
2021-03-19 07:21:09.970402 < 32 DataRow 1 22 '{random_page_cost=3.0}'
2021-03-19 07:21:09.970420 < 13 CommandComplete "SELECT 1"
2021-03-19 07:21:09.970431 < 5 ReadyForQuery I
2021-03-19 07:21:09.970558 > 42 Query "DROP TABLESPACE regress_tblspacewith;"
2021-03-19 07:21:09.971500 < 20 CommandComplete "DROP TABLESPACE"
2021-03-19 07:21:09.971561 < 5 ReadyForQuery I
```

-----Original Message-----
From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Friday, March 19, 2021 11:37 AM

Thanks to removing the static arrays, the code got much smaller. I'm happy
with this result.

Thank you so much. Your advice was very helpful in refactoring the patch.

(1)
+      (<literal>&gt;</literal> for messages from client to server,
+      and <literal>&lt;</literal> for messages from server to client),

I think this was meant to say "> or <". So, maybe you should remove "," at
the end of the first line, and replace "and" with "or".

I fixed.

(2)
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messsage does not reach here. */

messsage does -> messages do

I fixed.

(3)
+ fprintf(f, "\tAuthentication");

Why don't you move this \t in each message type to the end of:

+ fprintf(conn->Pfdebug, "%s\t%s\t%d", timestr, prefix, length);

I fixed.

Plus, don't we say in the manual that fields (timestamp, direction, length,
message type, and message content) are separated by a tab?

Sure. I added following documentation;
+ Non-message contents fields (timestamp, direction, length and message type)
+ are separated by a tab. Message contents are separated by a space.

Also, the code doesn't seem to separate the message type and content with a
tab.

I fixed to output a tab at the end of message types.

(4)
Where did the processing for unknown message go in
pqTraceOutputMessage()? Add default label?

(5)
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug,
"%s\t>\t%d\tCancelRequest", timestr, length);

I think this should follow pqTraceOutputMessage(). That is, each case label
only print the message type name in case other message types are added in
the future.

Sure. I fixed pqTraceOutputNoTypeByteMessage() following to pqTraceOutputMessage() style.

Regards,
Aya Iwata

Attachments:

v29-libpq-trace-log.patchapplication/octet-stream; name=v29-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index be674fb..838e394 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6459,12 +6459,27 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server
+      or <literal>&lt;</literal> for messages from server to client),
+      message length, message type, and message contents.
+      Non-message contents fields (timestamp, direction, length and message type)
+      are separated by a tab. Message contents are separated by a space.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -6479,6 +6494,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..0424523 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5c48c14..a00701f 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -183,3 +183,4 @@ PQenterPipelineMode       180
 PQexitPipelineMode        181
 PQpipelineSync            182
 PQpipelineStatus          183
+PQtraceSetFlags           184
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 53b354a..a90bdb8 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6859,27 +6859,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index f344370..19e8a4b 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -970,10 +970,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..54d6a76 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -84,9 +84,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +97,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +132,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +157,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +175,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +192,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +207,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +244,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +277,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +485,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +519,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +528,16 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +964,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 306e89a..de77c06 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -457,6 +457,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1660,6 +1663,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2121,6 +2127,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index cee42d4..d0decde 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -376,7 +376,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6374ec6..1a1d0e1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -818,6 +819,12 @@ extern ssize_t pg_GSS_write(PGconn *conn, const void *ptr, size_t len);
 extern ssize_t pg_GSS_read(PGconn *conn, void *ptr, size_t len);
 #endif
 
+/* === in libpq-trace.c === */
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message,
+															bool toServer);
+extern void pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message);
+
 /* === miscellaneous macros === */
 
 /*
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..bacb790
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,717 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+{
+	const char *v = data + *cursor;
+
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*v))
+		fprintf(pfdebug, " \\x%02x", *v);
+	else
+		fprintf(pfdebug, " %c", *v);
+	++*cursor;
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+{
+	uint16		tmp;
+	int			result;
+
+	memcpy(&tmp, data + *cursor , 2);
+	*cursor += 2;
+	result = (int) pg_ntoh16(tmp);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data + *cursor, 4);
+	*cursor += 4;
+	result = (int) pg_ntoh32(result);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
+{
+	int	len;
+
+	len = fprintf(pfdebug, " \"%s\"", data + *cursor);
+
+	/*
+	 * This is null-terminated string. So add 1 after subtracting 3
+	 * which is the double quotes and space length from len.
+	 */
+	*cursor += (len - 3 + 1);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+{
+	int			i,
+				next;			/* first char not yet printed */
+	const char	*v = data + *cursor;
+
+	fprintf(pfdebug, " \'");
+
+	for (next = i = 0; i < len; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < len)
+		fwrite(v + next, 1, len - next, pfdebug);
+
+	fprintf(pfdebug, "\'");
+	*cursor += len;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "Authentication\t");
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "BackendKeyData\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(const char *message, int end, FILE *f)
+{
+	int cursor = 0;
+	int nparams;
+	int nbytes;
+	int i;
+
+	fprintf(f, "Bind\t");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "Close\t");
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, f);
+	}
+	else
+	{
+		fprintf(f, "CommandComplete\t");
+		pqTraceOutputString(message, &cursor, f);
+	}
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "CopyFail\t");
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "CopyInResponse\t");
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "CopyOutResponse\t");
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "CopyBothResponse\t");
+	pqTraceOutputByte1(message, &cursor, f);
+
+	while (end > cursor)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "Describe\t");
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, f);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		fprintf(f, "DataRow\t");
+		nfields = pqTraceOutputInt16(message, &cursor, f);
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message, &cursor, f);
+			if (len == -1)
+				continue;
+			pqTraceOutputNchar(message, &cursor, f, len);
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "Execute\t");
+		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+	}
+	else
+	{
+		fprintf(f, "ErrorResponse\t");
+		while (end > cursor)
+		{
+			pqTraceOutputByte1(message, &cursor, f);
+			if (message[cursor] == '\0')
+				continue;
+			pqTraceOutputString(message, &cursor, f);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int nbytes;
+	int	i;
+
+	fprintf(f, "FunctionCall\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int		len;
+
+	fprintf(f, "FunctionCallResponse\t");
+	len = pqTraceOutputInt32(message, &cursor, f);
+	if (len != -1)
+		pqTraceOutputNchar(message, &cursor, f, len);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "NegotiateProtocolVersion\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "NoticeResponse\t");
+	while (end > cursor)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		if (message[cursor] == '\0')
+			continue;
+		pqTraceOutputString(message, &cursor, f);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "NotificationResponse\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "ParameterDescription\t");
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "ParameterStatus\t");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* Parse */
+static void
+pqTraceOutputP(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nparams;
+	int i;
+
+	fprintf(f, "Parse\t");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Query */
+static void
+pqTraceOutputQ(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "Query\t");
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "ReadyForQuery\t");
+	pqTraceOutputByte1(message, &cursor, f);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	fprintf(f, "RowDescription\t");
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	char		timestr[128];
+	char 		id;
+	int			length;
+	char	   *prefix = toServer ? ">" : "<";
+	int			LogCursor = 0;
+	int			LogEnd;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%d\t", timestr, prefix, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (!toServer)
+				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+			else
+				fprintf(conn->Pfdebug, "Flush");
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'E':	/* Execute(F) or Error Response(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (!toServer)
+				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+			else
+				fprintf(conn->Pfdebug, "Sync");
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case '2':	/* Bind Complete */
+			fprintf(conn->Pfdebug, "BindComplete");
+			/* No message content */
+			break;
+		case '3':	/* Close Complete */
+			fprintf(conn->Pfdebug, "CloseComplete");
+			/* No message content */
+			break;
+		case 'c':	/* Copy Done */
+			fprintf(conn->Pfdebug, "CopyDone");
+			/* No message content */
+			break;
+		case 'I':	/* Empty Query Response */
+			fprintf(conn->Pfdebug, "EmptyQueryResponse");
+			/* No message content */
+			break;
+		case 'n':	/* No Data */
+			fprintf(conn->Pfdebug, "NoData");
+			/* No message content */
+			break;
+		case '1':	/* Parse Complete */
+			fprintf(conn->Pfdebug, "ParseComplete");
+			/* No message content */
+			break;
+		case 's':	/* Portal Suspended */
+			fprintf(conn->Pfdebug, "PortalSuspended");
+			/* No message content */
+			break;
+		case 'X':	/* Terminate */
+			fprintf(conn->Pfdebug, "Terminate");
+			/* No message content */
+			break;
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: %02x", id);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
+
+void
+pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
+{
+	char		timestr[128];
+	int			length;
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	fprintf(conn->Pfdebug, "%s\t>\t%d\t", timestr, length);
+
+	switch(length)
+	{
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "CancelRequest\t");
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			break;
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messages do not reach here. */
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: length is %d", length);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
#180tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: iwata.aya@fujitsu.com (#179)
RE: libpq debug log

Alvaro-san, Iwata-san,
cc: Tom-san, Horiguchi-san,

Thank you, it finally looks complete to me!

Alvaro-san,
We appreciate your help and patience, sometimes rewriting large part of the patch. Could you do the final check (and possibly tweak) for commit?

Regards
Takayuki Tsunakawa

#181alvherre@alvh.no-ip.org
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#180)
Re: libpq debug log

Hello

I added an option to the new libpq_pipeline program that it activates
libpq trace. It works nicely and I think we can add that to the
regression tests. However I have two observations.

1. The trace output for the error message won't be very nice, because it
prints line numbers. So if I want to match the output to an "expected"
file, it would break every time somebody edits the source file where the
error originates and the ereport() line is moved. For example:

< 70 ErrorResponse S "ERROR" V "ERROR" C "22012" M "division by zero" F "numeric.c" L "8366" R "div_var" \x00 "Z"

The line number 8366 in this line would be problematic. I think if we
want regression testing for this, we should have another trace flag to
suppress output of a few of these fields. I would have it print only S,
C and M.

(Hey, what the heck is that "Z" at the end of the message? I thought
the error ended right at the \x00 ...)

2. The < and > characters are not good for visual inspection. I
replaced them with F and B and I think it's much clearer. Compare:

27 Query "SET lc_messages TO "C""

< 8 CommandComplete "SET"
< 5 ReadyForQuery I

21 Parse "" "SELECT $1" 1 23
19 Bind "" "" 0 1 1 '1' 1 0
6 Describe P ""
9 Execute "" 0
4 Sync

< 4 ParseComplete
< 4 BindComplete
< 33 RowDescription 1 "?column?" 0 0 23 4 -1 0
< 11 DataRow 1 1 '1'
< 13 CommandComplete "SELECT 1"
< 5 ReadyForQuery I

4 Terminate

with

F 27 Query "SET lc_messages TO "C""
B 8 CommandComplete "SET"
B 5 ReadyForQuery I
F 21 Parse "" "SELECT $1" 1 23
F 19 Bind "" "" 0 1 1 '1' 1 0
F 6 Describe P ""
F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
B 33 RowDescription 1 "?column?" 0 0 23 4 -1 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
F 4 Terminate

I think the second one is much easier on the eye.

(This one is the output from "libpq_pipeline simple_pipeline").

--
�lvaro Herrera Valdivia, Chile
"Saca el libro que tu religi�n considere como el indicado para encontrar la
oraci�n que traiga paz a tu alma. Luego rebootea el computador
y ve si funciona" (Carlos Ducl�s)

#182alvherre@alvh.no-ip.org
alvherre@alvh.no-ip.org
In reply to: alvherre@alvh.no-ip.org (#181)
5 attachment(s)
Re: libpq debug log

Proposed changes on top of v29.

--
�lvaro Herrera Valdivia, Chile

Attachments:

0001-libpq_pipeline-add-t-support-for-PQtrace.patchtext/x-diff; charset=us-asciiDownload
From b32ae3805bb28553c0a1cf308c6ed27f58576f3c Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 19:12:12 -0300
Subject: [PATCH 1/5] libpq_pipeline: add -t support for PQtrace

---
 .../modules/libpq_pipeline/libpq_pipeline.c   | 84 +++++++++++++------
 1 file changed, 59 insertions(+), 25 deletions(-)

diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 846ee9f5ab..34d085035b 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -23,6 +23,7 @@
 #include "catalog/pg_type_d.h"
 #include "common/fe_memutils.h"
 #include "libpq-fe.h"
+#include "pg_getopt.h"
 #include "portability/instr_time.h"
 
 
@@ -30,6 +31,9 @@ static void exit_nicely(PGconn *conn);
 
 const char *const progname = "libpq_pipeline";
 
+/* Options and defaults */
+char	   *tracefile = NULL;	/* path to PQtrace() file */
+
 
 #define DEBUG
 #ifdef DEBUG
@@ -1209,8 +1213,10 @@ usage(const char *progname)
 {
 	fprintf(stderr, "%s tests libpq's pipeline mode.\n\n", progname);
 	fprintf(stderr, "Usage:\n");
-	fprintf(stderr, "  %s tests", progname);
-	fprintf(stderr, "  %s testname [conninfo [number_of_rows]]\n", progname);
+	fprintf(stderr, "  %s [OPTION] tests\n", progname);
+	fprintf(stderr, "  %s [OPTION] TESTNAME [CONNINFO [NUMBER_OF_ROWS]\n", progname);
+	fprintf(stderr, "\nOptions:\n");
+	fprintf(stderr, "  -t TRACEFILE       generate a libpq trace to TRACEFILE\n");
 }
 
 static void
@@ -1231,37 +1237,54 @@ main(int argc, char **argv)
 {
 	const char *conninfo = "";
 	PGconn	   *conn;
+	FILE	   *trace;
+	char	   *testname;
 	int			numrows = 10000;
 	PGresult   *res;
+	int			c;
 
-	if (strcmp(argv[1], "tests") == 0)
+	while ((c = getopt(argc, argv, "t:")) != -1)
 	{
-		print_test_list();
-		exit(0);
+		switch (c)
+		{
+			case 't':			/* trace file */
+				tracefile = pg_strdup(optarg);
+				break;
+		}
 	}
 
-	/*
-	 * The testname parameter is mandatory; it can be followed by a conninfo
-	 * string and number of rows.
-	 */
-	if (argc < 2 || argc > 4)
+	if (optind < argc)
+	{
+		testname = argv[optind];
+		optind++;
+	}
+	else
 	{
 		usage(argv[0]);
 		exit(1);
 	}
 
-	if (argc >= 3)
-		conninfo = pg_strdup(argv[2]);
+	if (strcmp(testname, "tests") == 0)
+	{
+		print_test_list();
+		exit(0);
+	}
 
-	if (argc >= 4)
+	if (optind < argc)
+	{
+		conninfo = argv[optind];
+		optind++;
+	}
+	if (optind < argc)
 	{
 		errno = 0;
-		numrows = strtol(argv[3], NULL, 10);
+		numrows = strtol(argv[optind], NULL, 10);
 		if (errno != 0 || numrows <= 0)
 		{
-			fprintf(stderr, "couldn't parse \"%s\" as a positive integer\n", argv[3]);
+			fprintf(stderr, "couldn't parse \"%s\" as a positive integer\n", argv[optind]);
 			exit(1);
 		}
+		optind++;
 	}
 
 	/* Make a connection to the database */
@@ -1272,30 +1295,41 @@ main(int argc, char **argv)
 				PQerrorMessage(conn));
 		exit_nicely(conn);
 	}
+
+	/* Set the trace file, if requested */
+	if (tracefile != NULL)
+	{
+		trace = fopen(tracefile, "w+");
+
+		if (trace == NULL)
+			pg_fatal("could not open file \"%s\": %m", tracefile);
+		PQtrace(conn, trace);
+		PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);
+	}
+
 	res = PQexec(conn, "SET lc_messages TO \"C\"");
 	if (PQresultStatus(res) != PGRES_COMMAND_OK)
 		pg_fatal("failed to set lc_messages: %s", PQerrorMessage(conn));
 
-	if (strcmp(argv[1], "disallowed_in_pipeline") == 0)
+	if (strcmp(testname, "disallowed_in_pipeline") == 0)
 		test_disallowed_in_pipeline(conn);
-	else if (strcmp(argv[1], "multi_pipelines") == 0)
+	else if (strcmp(testname, "multi_pipelines") == 0)
 		test_multi_pipelines(conn);
-	else if (strcmp(argv[1], "pipeline_abort") == 0)
+	else if (strcmp(testname, "pipeline_abort") == 0)
 		test_pipeline_abort(conn);
-	else if (strcmp(argv[1], "pipelined_insert") == 0)
+	else if (strcmp(testname, "pipelined_insert") == 0)
 		test_pipelined_insert(conn, numrows);
-	else if (strcmp(argv[1], "prepared") == 0)
+	else if (strcmp(testname, "prepared") == 0)
 		test_prepared(conn);
-	else if (strcmp(argv[1], "simple_pipeline") == 0)
+	else if (strcmp(testname, "simple_pipeline") == 0)
 		test_simple_pipeline(conn);
-	else if (strcmp(argv[1], "singlerow") == 0)
+	else if (strcmp(testname, "singlerow") == 0)
 		test_singlerowmode(conn);
-	else if (strcmp(argv[1], "transaction") == 0)
+	else if (strcmp(testname, "transaction") == 0)
 		test_transaction(conn);
 	else
 	{
-		fprintf(stderr, "\"%s\" is not a recognized test name\n", argv[1]);
-		usage(argv[0]);
+		fprintf(stderr, "\"%s\" is not a recognized test name\n", testname);
 		exit(1);
 	}
 
-- 
2.20.1

0002-put-FILE-arg-always-first.patchtext/x-diff; charset=us-asciiDownload
From 4b62c3159fe3aa8445317a5d65b7e81d91c7fba6 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 19:13:04 -0300
Subject: [PATCH 2/5] put FILE * arg always first

---
 src/interfaces/libpq/libpq-trace.c | 234 ++++++++++++++---------------
 1 file changed, 117 insertions(+), 117 deletions(-)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index bacb7903b9..bff542cada 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -93,7 +93,7 @@ pqTraceFormatTimestamp(char *timestr, size_t ts_len)
  *   pqTraceOutputByte1: output 1 char message to the log
  */
 static void
-pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputByte1(FILE *pfdebug, const char *data, int *cursor)
 {
 	const char *v = data + *cursor;
 
@@ -113,7 +113,7 @@ pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
  *   pqTraceOutputInt16: output a 2-byte integer message to the log
  */
 static int
-pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputInt16(FILE *pfdebug, const char *data, int *cursor)
 {
 	uint16		tmp;
 	int			result;
@@ -130,7 +130,7 @@ pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
  *   pqTraceOutputInt32: output a 4-byte integer message to the log
  */
 static int
-pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputInt32(FILE *pfdebug, const char *data, int *cursor)
 {
 	int			result;
 
@@ -146,7 +146,7 @@ pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
  *   pqTraceOutputString: output a string message to the log
  */
 static void
-pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputString(FILE *pfdebug, const char *data, int *cursor)
 {
 	int	len;
 
@@ -163,7 +163,7 @@ pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
  * pqTraceOutputNchar: output a string of exactly len bytes message to the log
  */
 static void
-pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+pqTraceOutputNchar(FILE *pfdebug, int len, const char *data, int *cursor)
 {
 	int			i,
 				next;			/* first char not yet printed */
@@ -195,28 +195,28 @@ pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
 
 /* Authentication */
 static void
-pqTraceOutputR(const char *message, FILE *f)
+pqTraceOutputR(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "Authentication\t");
-	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* BackendKeyData */
 static void
-pqTraceOutputK(const char *message, FILE *f)
+pqTraceOutputK(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "BackendKeyData\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* Bind */
 static void
-pqTraceOutputB(const char *message, int end, FILE *f)
+pqTraceOutputB(FILE *f, const char *message, int end)
 {
 	int cursor = 0;
 	int nparams;
@@ -224,113 +224,113 @@ pqTraceOutputB(const char *message, int end, FILE *f)
 	int i;
 
 	fprintf(f, "Bind\t");
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nparams; i++)
 	{
-		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		nbytes = pqTraceOutputInt32(f, message, &cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(message, &cursor, f, nbytes);
+		pqTraceOutputNchar(f, nbytes, message, &cursor);
 	}
 
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* Close(F) or CommandComplete(B) */
 static void
-pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+pqTraceOutputC(FILE *f, bool toServer, const char *message, int end)
 {
 	int	cursor = 0;
 
 	if (toServer)
 	{
 		fprintf(f, "Close\t");
-		pqTraceOutputByte1(message, &cursor, f);
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputByte1(f, message, &cursor);
+		pqTraceOutputString(f, message, &cursor);
 	}
 	else
 	{
 		fprintf(f, "CommandComplete\t");
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
 	}
 }
 
 /* CopyFail */
 static void
-pqTraceOutputf(const char *message, int end, FILE *f)
+pqTraceOutputf(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "CopyFail\t");
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* CopyInResponse */
 static void
-pqTraceOutputG(const char *message, int end, FILE *f)
+pqTraceOutputG(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int	nfields;
 	int	i;
 
 	fprintf(f, "CopyInResponse\t");
-	pqTraceOutputByte1(message, &cursor, f);
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* CopyOutResponse */
 static void
-pqTraceOutputH(const char *message, int end, FILE *f)
+pqTraceOutputH(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int	nfields;
 	int	i;
 
 	fprintf(f, "CopyOutResponse\t");
-	pqTraceOutputByte1(message, &cursor, f);
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* CopyBothResponse */
 static void
-pqTraceOutputW(const char *message, int end, FILE *f)
+pqTraceOutputW(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "CopyBothResponse\t");
-	pqTraceOutputByte1(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
 
 	while (end > cursor)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* Describe(F) or DataRow(B) */
 static void
-pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+pqTraceOutputD(FILE *f, bool toServer, const char *message, int end)
 {
 	int	cursor = 0;
 
 	if (toServer)
 	{
 		fprintf(f, "Describe\t");
-		pqTraceOutputByte1(message, &cursor, f);
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputByte1(f, message, &cursor);
+		pqTraceOutputString(f, message, &cursor);
 	}
 	else
 	{
@@ -339,45 +339,45 @@ pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
 		int		i;
 
 		fprintf(f, "DataRow\t");
-		nfields = pqTraceOutputInt16(message, &cursor, f);
+		nfields = pqTraceOutputInt16(f, message, &cursor);
 		for (i = 0; i < nfields; i++)
 		{
-			len = pqTraceOutputInt32(message, &cursor, f);
+			len = pqTraceOutputInt32(f, message, &cursor);
 			if (len == -1)
 				continue;
-			pqTraceOutputNchar(message, &cursor, f, len);
+			pqTraceOutputNchar(f, len, message, &cursor);
 		}
 	}
 }
 
 /* Execute(F) or ErrorResponse(B) */
 static void
-pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+pqTraceOutputE(FILE *f, bool toServer, const char *message, int end)
 {
 	int	cursor = 0;
 
 	if (toServer)
 	{
 		fprintf(f, "Execute\t");
-		pqTraceOutputString(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
 	}
 	else
 	{
 		fprintf(f, "ErrorResponse\t");
 		while (end > cursor)
 		{
-			pqTraceOutputByte1(message, &cursor, f);
+			pqTraceOutputByte1(f, message, &cursor);
 			if (message[cursor] == '\0')
 				continue;
-			pqTraceOutputString(message, &cursor, f);
+			pqTraceOutputString(f, message, &cursor);
 		}
 	}
 }
 
 /* FunctionCall */
 static void
-pqTraceOutputF(const char *message, FILE *f)
+pqTraceOutputF(FILE *f, const char *message)
 {
 	int	cursor = 0;
 	int nfields;
@@ -385,160 +385,160 @@ pqTraceOutputF(const char *message, FILE *f)
 	int	i;
 
 	fprintf(f, "FunctionCall\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
 	{
-		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		nbytes = pqTraceOutputInt32(f, message, &cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(message, &cursor, f, nbytes);
+		pqTraceOutputNchar(f, nbytes, message, &cursor);
 	}
 
-	pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* FunctionCallResponse */
 static void
-pqTraceOutputV(const char *message, FILE *f)
+pqTraceOutputV(FILE *f, const char *message)
 {
 	int	cursor = 0;
 	int		len;
 
 	fprintf(f, "FunctionCallResponse\t");
-	len = pqTraceOutputInt32(message, &cursor, f);
+	len = pqTraceOutputInt32(f, message, &cursor);
 	if (len != -1)
-		pqTraceOutputNchar(message, &cursor, f, len);
+		pqTraceOutputNchar(f, len, message, &cursor);
 }
 
 /* NegotiateProtocolVersion */
 static void
-pqTraceOutputv(const char *message, FILE *f)
+pqTraceOutputv(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "NegotiateProtocolVersion\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* NoticeResponse */
 static void
-pqTraceOutputN(const char *message, int end, FILE *f)
+pqTraceOutputN(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "NoticeResponse\t");
 	while (end > cursor)
 	{
-		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputByte1(f, message, &cursor);
 		if (message[cursor] == '\0')
 			continue;
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
 	}
 }
 
 /* NotificationResponse */
 static void
-pqTraceOutputA(const char *message, int end, FILE *f)
+pqTraceOutputA(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "NotificationResponse\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* ParameterDescription */
 static void
-pqTraceOutputt(const char *message, FILE *f)
+pqTraceOutputt(FILE *f, const char *message)
 {
 	int	cursor = 0;
 	int	nfields;
 	int	i;
 
 	fprintf(f, "ParameterDescription\t");
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* ParameterStatus */
 static void
-pqTraceOutputS(const char *message, int end, FILE *f)
+pqTraceOutputS(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "ParameterStatus\t");
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* Parse */
 static void
-pqTraceOutputP(const char *message, int end, FILE *f)
+pqTraceOutputP(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int nparams;
 	int i;
 
 	fprintf(f, "Parse\t");
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* Query */
 static void
-pqTraceOutputQ(const char *message, int end, FILE *f)
+pqTraceOutputQ(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "Query\t");
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* ReadyForQuery */
 static void
-pqTraceOutputZ(const char *message, FILE *f)
+pqTraceOutputZ(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "ReadyForQuery\t");
-	pqTraceOutputByte1(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
 }
 
 /* RowDescription */
 static void
-pqTraceOutputT(const char *message, int end, FILE *f)
+pqTraceOutputT(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int nfields;
 	int	i;
 
 	fprintf(f, "RowDescription\t");
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
 	{
-		pqTraceOutputString(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
-		pqTraceOutputInt16(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
-		pqTraceOutputInt16(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputInt16(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputInt16(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputInt16(f, message, &cursor);
 	}
 }
 
@@ -569,76 +569,76 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	switch(id)
 	{
 		case 'R':	/* Authentication */
-			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputR(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'K':	/* secret key data from the backend */
-			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputK(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'B':	/* Bind */
-			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputB(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'C':	/* Close(F) or Command Complete(B) */
-			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			pqTraceOutputC(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
 			break;
 		case 'd':	/* Copy Data */
 			/* Drop COPY data to reduce the overhead of logging. */
 			break;
 		case 'f':	/* Copy Fail */
-			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputf(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'G':	/* Start Copy In */
-			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputG(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'H':	/* Flush(F) or Start Copy Out(B) */
 			if (!toServer)
-				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+				pqTraceOutputH(conn->Pfdebug, message + LogCursor, LogEnd);
 			else
 				fprintf(conn->Pfdebug, "Flush");
 			break;
 		case 'W':	/* Start Copy Both */
-			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputW(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'D':	/* Describe(F) or Data Row(B) */
-			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			pqTraceOutputD(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
 			break;
 		case 'E':	/* Execute(F) or Error Response(B) */
-			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			pqTraceOutputE(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
 			break;
 		case 'F':	/* Function Call */
-			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputF(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'V':	/* Function Call response */
-			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputV(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'v':	/* Negotiate Protocol Version */
-			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputv(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'N':	/* Notice Response */
-			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputN(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'A':	/* Notification Response */
-			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputA(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 't':	/* Parameter Description */
-			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputt(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'S':	/* Parameter Status(B) or Sync(F) */
 			if (!toServer)
-				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+				pqTraceOutputS(conn->Pfdebug, message + LogCursor, LogEnd);
 			else
 				fprintf(conn->Pfdebug, "Sync");
 			break;
 		case 'P':	/* Parse */
-			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputP(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'Q':	/* Query */
-			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputQ(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'Z':	/* Ready For Query */
-			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputZ(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'T':	/* Row Description */
-			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputT(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case '2':	/* Bind Complete */
 			fprintf(conn->Pfdebug, "BindComplete");
@@ -702,9 +702,9 @@ pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
 	{
 		case 16:	/* CancelRequest */
 			fprintf(conn->Pfdebug, "CancelRequest\t");
-			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
-			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
-			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 8 :	/* GSSENCRequest or SSLRequest */
 			/* These messages do not reach here. */
-- 
2.20.1

0003-use-F-B-instead-of.patchtext/x-diff; charset=us-asciiDownload
From d44009a25de3af549dfa528a4fa44c9ebf785c42 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 19:13:16 -0300
Subject: [PATCH 3/5] use F/B instead of </>

---
 src/interfaces/libpq/libpq-trace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index bff542cada..94f28f04a8 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -548,7 +548,7 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	char		timestr[128];
 	char 		id;
 	int			length;
-	char	   *prefix = toServer ? ">" : "<";
+	char	   *prefix = toServer ? "F" : "B";
 	int			LogCursor = 0;
 	int			LogEnd;
 
-- 
2.20.1

0004-pass-cursor-to-printing-routines.patchtext/x-diff; charset=us-asciiDownload
From 39a5e0c6b5559a0fb9a6c376e493e4ee4bc980fb Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 20:04:53 -0300
Subject: [PATCH 4/5] pass *cursor to printing routines

---
 src/interfaces/libpq/libpq-trace.c | 309 +++++++++++++----------------
 1 file changed, 134 insertions(+), 175 deletions(-)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index 94f28f04a8..06d8d981a3 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -106,7 +106,7 @@ pqTraceOutputByte1(FILE *pfdebug, const char *data, int *cursor)
 		fprintf(pfdebug, " \\x%02x", *v);
 	else
 		fprintf(pfdebug, " %c", *v);
-	++*cursor;
+	*cursor += 1;
 }
 
 /*
@@ -195,142 +195,125 @@ pqTraceOutputNchar(FILE *pfdebug, int len, const char *data, int *cursor)
 
 /* Authentication */
 static void
-pqTraceOutputR(FILE *f, const char *message)
+pqTraceOutputR(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "Authentication\t");
-	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
 }
 
 /* BackendKeyData */
 static void
-pqTraceOutputK(FILE *f, const char *message)
+pqTraceOutputK(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "BackendKeyData\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputInt32(f, message, cursor);
 }
 
 /* Bind */
 static void
-pqTraceOutputB(FILE *f, const char *message, int end)
+pqTraceOutputB(FILE *f, const char *message, int *cursor)
 {
-	int cursor = 0;
 	int nparams;
-	int nbytes;
-	int i;
 
 	fprintf(f, "Bind\t");
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
-	nparams = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt16(f, message, cursor);
 
-	nparams = pqTraceOutputInt16(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nparams; i++)
+	for (int i = 0; i < nparams; i++)
 	{
-		nbytes = pqTraceOutputInt32(f, message, &cursor);
+		int nbytes;
+
+		nbytes = pqTraceOutputInt32(f, message, cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(f, nbytes, message, &cursor);
+		pqTraceOutputNchar(f, nbytes, message, cursor);
 	}
 
-	nparams = pqTraceOutputInt16(f, message, &cursor);
-	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* Close(F) or CommandComplete(B) */
 static void
-pqTraceOutputC(FILE *f, bool toServer, const char *message, int end)
+pqTraceOutputC(FILE *f, bool toServer, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	if (toServer)
 	{
 		fprintf(f, "Close\t");
-		pqTraceOutputByte1(f, message, &cursor);
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputByte1(f, message, cursor);
+		pqTraceOutputString(f, message, cursor);
 	}
 	else
 	{
 		fprintf(f, "CommandComplete\t");
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputString(f, message, cursor);
 	}
 }
 
 /* CopyFail */
 static void
-pqTraceOutputf(FILE *f, const char *message, int end)
+pqTraceOutputf(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "CopyFail\t");
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* CopyInResponse */
 static void
-pqTraceOutputG(FILE *f, const char *message, int end)
+pqTraceOutputG(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int	nfields;
-	int	i;
 
 	fprintf(f, "CopyInResponse\t");
-	pqTraceOutputByte1(f, message, &cursor);
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* CopyOutResponse */
 static void
-pqTraceOutputH(FILE *f, const char *message, int end)
+pqTraceOutputH(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int	nfields;
-	int	i;
 
 	fprintf(f, "CopyOutResponse\t");
-	pqTraceOutputByte1(f, message, &cursor);
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* CopyBothResponse */
 static void
-pqTraceOutputW(FILE *f, const char *message, int end)
+pqTraceOutputW(FILE *f, const char *message, int *cursor, int length)
 {
-	int	cursor = 0;
-
 	fprintf(f, "CopyBothResponse\t");
-	pqTraceOutputByte1(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
 
-	while (end > cursor)
-		pqTraceOutputInt16(f, message, &cursor);
+	while (length > *cursor)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* Describe(F) or DataRow(B) */
 static void
-pqTraceOutputD(FILE *f, bool toServer, const char *message, int end)
+pqTraceOutputD(FILE *f, bool toServer, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	if (toServer)
 	{
 		fprintf(f, "Describe\t");
-		pqTraceOutputByte1(f, message, &cursor);
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputByte1(f, message, cursor);
+		pqTraceOutputString(f, message, cursor);
 	}
 	else
 	{
@@ -339,206 +322,183 @@ pqTraceOutputD(FILE *f, bool toServer, const char *message, int end)
 		int		i;
 
 		fprintf(f, "DataRow\t");
-		nfields = pqTraceOutputInt16(f, message, &cursor);
+		nfields = pqTraceOutputInt16(f, message, cursor);
 		for (i = 0; i < nfields; i++)
 		{
-			len = pqTraceOutputInt32(f, message, &cursor);
+			len = pqTraceOutputInt32(f, message, cursor);
 			if (len == -1)
 				continue;
-			pqTraceOutputNchar(f, len, message, &cursor);
+			pqTraceOutputNchar(f, len, message, cursor);
 		}
 	}
 }
 
 /* Execute(F) or ErrorResponse(B) */
 static void
-pqTraceOutputE(FILE *f, bool toServer, const char *message, int end)
+pqTraceOutputE(FILE *f, bool toServer, const char *message, int *cursor, int length)
 {
-	int	cursor = 0;
-
 	if (toServer)
 	{
 		fprintf(f, "Execute\t");
-		pqTraceOutputString(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputString(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
 	}
 	else
 	{
 		fprintf(f, "ErrorResponse\t");
-		while (end > cursor)
+		for (;;)
 		{
-			pqTraceOutputByte1(f, message, &cursor);
-			if (message[cursor] == '\0')
-				continue;
-			pqTraceOutputString(f, message, &cursor);
+			pqTraceOutputByte1(f, message, cursor);
+			if (message[*cursor - 1] == '\0')
+				break;
+			pqTraceOutputString(f, message, cursor);
 		}
 	}
 }
 
 /* FunctionCall */
 static void
-pqTraceOutputF(FILE *f, const char *message)
+pqTraceOutputF(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int nfields;
 	int nbytes;
-	int	i;
 
 	fprintf(f, "FunctionCall\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
 
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
+	for (int i = 0; i < nfields; i++)
 	{
-		nbytes = pqTraceOutputInt32(f, message, &cursor);
+		nbytes = pqTraceOutputInt32(f, message, cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(f, nbytes, message, &cursor);
+		pqTraceOutputNchar(f, nbytes, message, cursor);
 	}
 
-	pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputInt16(f, message, cursor);
 }
 
 /* FunctionCallResponse */
 static void
-pqTraceOutputV(FILE *f, const char *message)
+pqTraceOutputV(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int		len;
 
 	fprintf(f, "FunctionCallResponse\t");
-	len = pqTraceOutputInt32(f, message, &cursor);
+	len = pqTraceOutputInt32(f, message, cursor);
 	if (len != -1)
-		pqTraceOutputNchar(f, len, message, &cursor);
+		pqTraceOutputNchar(f, len, message, cursor);
 }
 
 /* NegotiateProtocolVersion */
 static void
-pqTraceOutputv(FILE *f, const char *message)
+pqTraceOutputv(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "NegotiateProtocolVersion\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputInt32(f, message, cursor);
 }
 
 /* NoticeResponse */
 static void
-pqTraceOutputN(FILE *f, const char *message, int end)
+pqTraceOutputN(FILE *f, const char *message, int *cursor, int length)
 {
-	int	cursor = 0;
-
 	fprintf(f, "NoticeResponse\t");
-	while (end > cursor)
+	for (;;)
 	{
-		pqTraceOutputByte1(f, message, &cursor);
-		if (message[cursor] == '\0')
-			continue;
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputByte1(f, message, cursor);
+		if (message[*cursor - 1] == '\0')
+			break;
+		pqTraceOutputString(f, message, cursor);
 	}
 }
 
 /* NotificationResponse */
 static void
-pqTraceOutputA(FILE *f, const char *message, int end)
+pqTraceOutputA(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "NotificationResponse\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* ParameterDescription */
 static void
-pqTraceOutputt(FILE *f, const char *message)
+pqTraceOutputt(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int	nfields;
-	int	i;
 
 	fprintf(f, "ParameterDescription\t");
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt32(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt32(f, message, cursor);
 }
 
 /* ParameterStatus */
 static void
-pqTraceOutputS(FILE *f, const char *message, int end)
+pqTraceOutputS(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "ParameterStatus\t");
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* Parse */
 static void
-pqTraceOutputP(FILE *f, const char *message, int end)
+pqTraceOutputP(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int nparams;
-	int i;
 
 	fprintf(f, "Parse\t");
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
-	nparams = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt32(f, message, &cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt32(f, message, cursor);
 }
 
 /* Query */
 static void
-pqTraceOutputQ(FILE *f, const char *message, int end)
+pqTraceOutputQ(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "Query\t");
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* ReadyForQuery */
 static void
-pqTraceOutputZ(FILE *f, const char *message)
+pqTraceOutputZ(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "ReadyForQuery\t");
-	pqTraceOutputByte1(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
 }
 
 /* RowDescription */
 static void
-pqTraceOutputT(FILE *f, const char *message, int end)
+pqTraceOutputT(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int nfields;
-	int	i;
 
 	fprintf(f, "RowDescription\t");
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
+	for (int i = 0; i < nfields; i++)
 	{
-		pqTraceOutputString(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
-		pqTraceOutputInt16(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
-		pqTraceOutputInt16(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
-		pqTraceOutputInt16(f, message, &cursor);
+		pqTraceOutputString(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
 	}
 }
 
@@ -550,95 +510,94 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	int			length;
 	char	   *prefix = toServer ? "F" : "B";
 	int			LogCursor = 0;
-	int			LogEnd;
 
 	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
 		pqTraceFormatTimestamp(timestr, sizeof(timestr));
-	else
-		timestr[0] = '\0';
+		fprintf(conn->Pfdebug, "%s\t", timestr);
+	}
 
 	id = message[LogCursor++];
 
-	memcpy(&length, message + LogCursor , 4);
+	memcpy(&length, message + LogCursor, 4);
 	length = (int) pg_ntoh32(length);
 	LogCursor += 4;
-	LogEnd = length - 4;
 
-	fprintf(conn->Pfdebug, "%s\t%s\t%d\t", timestr, prefix, length);
+	fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
 
 	switch(id)
 	{
 		case 'R':	/* Authentication */
-			pqTraceOutputR(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputR(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'K':	/* secret key data from the backend */
-			pqTraceOutputK(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputK(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'B':	/* Bind */
-			pqTraceOutputB(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputB(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'C':	/* Close(F) or Command Complete(B) */
-			pqTraceOutputC(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
+			pqTraceOutputC(conn->Pfdebug, toServer, message, &LogCursor);
 			break;
 		case 'd':	/* Copy Data */
 			/* Drop COPY data to reduce the overhead of logging. */
 			break;
 		case 'f':	/* Copy Fail */
-			pqTraceOutputf(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputf(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'G':	/* Start Copy In */
-			pqTraceOutputG(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputG(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'H':	/* Flush(F) or Start Copy Out(B) */
 			if (!toServer)
-				pqTraceOutputH(conn->Pfdebug, message + LogCursor, LogEnd);
+				pqTraceOutputH(conn->Pfdebug, message, &LogCursor);
 			else
 				fprintf(conn->Pfdebug, "Flush");
 			break;
 		case 'W':	/* Start Copy Both */
-			pqTraceOutputW(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputW(conn->Pfdebug, message, &LogCursor, length);
 			break;
 		case 'D':	/* Describe(F) or Data Row(B) */
-			pqTraceOutputD(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
+			pqTraceOutputD(conn->Pfdebug, toServer, message, &LogCursor);
 			break;
 		case 'E':	/* Execute(F) or Error Response(B) */
-			pqTraceOutputE(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
+			pqTraceOutputE(conn->Pfdebug, toServer, message, &LogCursor, length);
 			break;
 		case 'F':	/* Function Call */
-			pqTraceOutputF(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputF(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'V':	/* Function Call response */
-			pqTraceOutputV(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputV(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'v':	/* Negotiate Protocol Version */
-			pqTraceOutputv(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputv(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'N':	/* Notice Response */
-			pqTraceOutputN(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputN(conn->Pfdebug, message, &LogCursor, length);
 			break;
 		case 'A':	/* Notification Response */
-			pqTraceOutputA(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputA(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 't':	/* Parameter Description */
-			pqTraceOutputt(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputt(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'S':	/* Parameter Status(B) or Sync(F) */
 			if (!toServer)
-				pqTraceOutputS(conn->Pfdebug, message + LogCursor, LogEnd);
+				pqTraceOutputS(conn->Pfdebug, message, &LogCursor);
 			else
 				fprintf(conn->Pfdebug, "Sync");
 			break;
 		case 'P':	/* Parse */
-			pqTraceOutputP(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputP(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'Q':	/* Query */
-			pqTraceOutputQ(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputQ(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'Z':	/* Ready For Query */
-			pqTraceOutputZ(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputZ(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'T':	/* Row Description */
-			pqTraceOutputT(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputT(conn->Pfdebug, message, &LogCursor);
 			break;
 		case '2':	/* Bind Complete */
 			fprintf(conn->Pfdebug, "BindComplete");
-- 
2.20.1

0005-add-length-check.patchtext/x-diff; charset=us-asciiDownload
From 4d645bdf08de419b207b0064986d81f758900809 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 20:05:10 -0300
Subject: [PATCH 5/5] add length check

---
 src/interfaces/libpq/libpq-trace.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index 06d8d981a3..6c58a578ac 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -637,6 +637,16 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	}
 
 	fputc('\n', conn->Pfdebug);
+
+	/*
+	 * Verify the printing routine did it right.  Note that the one-byte
+	 * message identifier is not included in the length, but our cursor
+	 * does include it.
+	 */
+	if (LogCursor - 1 != length)
+		fprintf(conn->Pfdebug,
+				"mismatched message length: consumed %d, expected %d\n",
+				LogCursor - 1, length);
 }
 
 void
-- 
2.20.1

#183alvherre@alvh.no-ip.org
alvherre@alvh.no-ip.org
In reply to: alvherre@alvh.no-ip.org (#182)
7 attachment(s)
Re: libpq debug log

On 2021-Mar-26, alvherre@alvh.no-ip.org wrote:

Proposed changes on top of v29.

This last one uses libpq_pipeline -t and verifies the output against an
expected trace file. Applies on top of all the previous patches. I
attach the whole lot, so that the CF bot has a chance to run it.

I did notice another problem for comparison of expected trace files,
which is that RowDescription includes table OIDs for some columns. I
think we would need to have a flag to suppress that too, somehow,
although the answer to what should we do is not as clear as for the
other two cases.

I dodged the issue by just using -t for the pipeline cases that don't
have OIDs in their output. This leaves us with no coverage for the
ErrorResponse message, which I think is a shortcoming. If we fixed the
OID problem and the ErrorResponse problem, we could add an expected
trace for pipeline_abort. I think we should do that.

Maybe the easiest way is to have a new flag PQTRACE_REGRESS_MODE.

--
�lvaro Herrera 39�49'30"S 73�17'W
Essentially, you're proposing Kevlar shoes as a solution for the problem
that you want to walk around carrying a loaded gun aimed at your foot.
(Tom Lane)

Attachments:

v30-0003-put-FILE-arg-always-first.patchtext/x-diff; charset=us-asciiDownload
From 4b62c3159fe3aa8445317a5d65b7e81d91c7fba6 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 19:13:04 -0300
Subject: [PATCH v30 3/7] put FILE * arg always first

---
 src/interfaces/libpq/libpq-trace.c | 234 ++++++++++++++---------------
 1 file changed, 117 insertions(+), 117 deletions(-)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index bacb7903b9..bff542cada 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -93,7 +93,7 @@ pqTraceFormatTimestamp(char *timestr, size_t ts_len)
  *   pqTraceOutputByte1: output 1 char message to the log
  */
 static void
-pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputByte1(FILE *pfdebug, const char *data, int *cursor)
 {
 	const char *v = data + *cursor;
 
@@ -113,7 +113,7 @@ pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
  *   pqTraceOutputInt16: output a 2-byte integer message to the log
  */
 static int
-pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputInt16(FILE *pfdebug, const char *data, int *cursor)
 {
 	uint16		tmp;
 	int			result;
@@ -130,7 +130,7 @@ pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
  *   pqTraceOutputInt32: output a 4-byte integer message to the log
  */
 static int
-pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputInt32(FILE *pfdebug, const char *data, int *cursor)
 {
 	int			result;
 
@@ -146,7 +146,7 @@ pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
  *   pqTraceOutputString: output a string message to the log
  */
 static void
-pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
+pqTraceOutputString(FILE *pfdebug, const char *data, int *cursor)
 {
 	int	len;
 
@@ -163,7 +163,7 @@ pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
  * pqTraceOutputNchar: output a string of exactly len bytes message to the log
  */
 static void
-pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+pqTraceOutputNchar(FILE *pfdebug, int len, const char *data, int *cursor)
 {
 	int			i,
 				next;			/* first char not yet printed */
@@ -195,28 +195,28 @@ pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
 
 /* Authentication */
 static void
-pqTraceOutputR(const char *message, FILE *f)
+pqTraceOutputR(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "Authentication\t");
-	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* BackendKeyData */
 static void
-pqTraceOutputK(const char *message, FILE *f)
+pqTraceOutputK(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "BackendKeyData\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* Bind */
 static void
-pqTraceOutputB(const char *message, int end, FILE *f)
+pqTraceOutputB(FILE *f, const char *message, int end)
 {
 	int cursor = 0;
 	int nparams;
@@ -224,113 +224,113 @@ pqTraceOutputB(const char *message, int end, FILE *f)
 	int i;
 
 	fprintf(f, "Bind\t");
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nparams; i++)
 	{
-		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		nbytes = pqTraceOutputInt32(f, message, &cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(message, &cursor, f, nbytes);
+		pqTraceOutputNchar(f, nbytes, message, &cursor);
 	}
 
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* Close(F) or CommandComplete(B) */
 static void
-pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+pqTraceOutputC(FILE *f, bool toServer, const char *message, int end)
 {
 	int	cursor = 0;
 
 	if (toServer)
 	{
 		fprintf(f, "Close\t");
-		pqTraceOutputByte1(message, &cursor, f);
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputByte1(f, message, &cursor);
+		pqTraceOutputString(f, message, &cursor);
 	}
 	else
 	{
 		fprintf(f, "CommandComplete\t");
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
 	}
 }
 
 /* CopyFail */
 static void
-pqTraceOutputf(const char *message, int end, FILE *f)
+pqTraceOutputf(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "CopyFail\t");
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* CopyInResponse */
 static void
-pqTraceOutputG(const char *message, int end, FILE *f)
+pqTraceOutputG(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int	nfields;
 	int	i;
 
 	fprintf(f, "CopyInResponse\t");
-	pqTraceOutputByte1(message, &cursor, f);
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* CopyOutResponse */
 static void
-pqTraceOutputH(const char *message, int end, FILE *f)
+pqTraceOutputH(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int	nfields;
 	int	i;
 
 	fprintf(f, "CopyOutResponse\t");
-	pqTraceOutputByte1(message, &cursor, f);
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* CopyBothResponse */
 static void
-pqTraceOutputW(const char *message, int end, FILE *f)
+pqTraceOutputW(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "CopyBothResponse\t");
-	pqTraceOutputByte1(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
 
 	while (end > cursor)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* Describe(F) or DataRow(B) */
 static void
-pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+pqTraceOutputD(FILE *f, bool toServer, const char *message, int end)
 {
 	int	cursor = 0;
 
 	if (toServer)
 	{
 		fprintf(f, "Describe\t");
-		pqTraceOutputByte1(message, &cursor, f);
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputByte1(f, message, &cursor);
+		pqTraceOutputString(f, message, &cursor);
 	}
 	else
 	{
@@ -339,45 +339,45 @@ pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
 		int		i;
 
 		fprintf(f, "DataRow\t");
-		nfields = pqTraceOutputInt16(message, &cursor, f);
+		nfields = pqTraceOutputInt16(f, message, &cursor);
 		for (i = 0; i < nfields; i++)
 		{
-			len = pqTraceOutputInt32(message, &cursor, f);
+			len = pqTraceOutputInt32(f, message, &cursor);
 			if (len == -1)
 				continue;
-			pqTraceOutputNchar(message, &cursor, f, len);
+			pqTraceOutputNchar(f, len, message, &cursor);
 		}
 	}
 }
 
 /* Execute(F) or ErrorResponse(B) */
 static void
-pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+pqTraceOutputE(FILE *f, bool toServer, const char *message, int end)
 {
 	int	cursor = 0;
 
 	if (toServer)
 	{
 		fprintf(f, "Execute\t");
-		pqTraceOutputString(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
 	}
 	else
 	{
 		fprintf(f, "ErrorResponse\t");
 		while (end > cursor)
 		{
-			pqTraceOutputByte1(message, &cursor, f);
+			pqTraceOutputByte1(f, message, &cursor);
 			if (message[cursor] == '\0')
 				continue;
-			pqTraceOutputString(message, &cursor, f);
+			pqTraceOutputString(f, message, &cursor);
 		}
 	}
 }
 
 /* FunctionCall */
 static void
-pqTraceOutputF(const char *message, FILE *f)
+pqTraceOutputF(FILE *f, const char *message)
 {
 	int	cursor = 0;
 	int nfields;
@@ -385,160 +385,160 @@ pqTraceOutputF(const char *message, FILE *f)
 	int	i;
 
 	fprintf(f, "FunctionCall\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt16(f, message, &cursor);
 
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
 	{
-		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		nbytes = pqTraceOutputInt32(f, message, &cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(message, &cursor, f, nbytes);
+		pqTraceOutputNchar(f, nbytes, message, &cursor);
 	}
 
-	pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputInt16(f, message, &cursor);
 }
 
 /* FunctionCallResponse */
 static void
-pqTraceOutputV(const char *message, FILE *f)
+pqTraceOutputV(FILE *f, const char *message)
 {
 	int	cursor = 0;
 	int		len;
 
 	fprintf(f, "FunctionCallResponse\t");
-	len = pqTraceOutputInt32(message, &cursor, f);
+	len = pqTraceOutputInt32(f, message, &cursor);
 	if (len != -1)
-		pqTraceOutputNchar(message, &cursor, f, len);
+		pqTraceOutputNchar(f, len, message, &cursor);
 }
 
 /* NegotiateProtocolVersion */
 static void
-pqTraceOutputv(const char *message, FILE *f)
+pqTraceOutputv(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "NegotiateProtocolVersion\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* NoticeResponse */
 static void
-pqTraceOutputN(const char *message, int end, FILE *f)
+pqTraceOutputN(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "NoticeResponse\t");
 	while (end > cursor)
 	{
-		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputByte1(f, message, &cursor);
 		if (message[cursor] == '\0')
 			continue;
-		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
 	}
 }
 
 /* NotificationResponse */
 static void
-pqTraceOutputA(const char *message, int end, FILE *f)
+pqTraceOutputA(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "NotificationResponse\t");
-	pqTraceOutputInt32(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* ParameterDescription */
 static void
-pqTraceOutputt(const char *message, FILE *f)
+pqTraceOutputt(FILE *f, const char *message)
 {
 	int	cursor = 0;
 	int	nfields;
 	int	i;
 
 	fprintf(f, "ParameterDescription\t");
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* ParameterStatus */
 static void
-pqTraceOutputS(const char *message, int end, FILE *f)
+pqTraceOutputS(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "ParameterStatus\t");
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* Parse */
 static void
-pqTraceOutputP(const char *message, int end, FILE *f)
+pqTraceOutputP(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int nparams;
 	int i;
 
 	fprintf(f, "Parse\t");
-	pqTraceOutputString(message, &cursor, f);
-	pqTraceOutputString(message, &cursor, f);
-	nparams = pqTraceOutputInt16(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt32(f, message, &cursor);
 }
 
 /* Query */
 static void
-pqTraceOutputQ(const char *message, int end, FILE *f)
+pqTraceOutputQ(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 
 	fprintf(f, "Query\t");
-	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(f, message, &cursor);
 }
 
 /* ReadyForQuery */
 static void
-pqTraceOutputZ(const char *message, FILE *f)
+pqTraceOutputZ(FILE *f, const char *message)
 {
 	int	cursor = 0;
 
 	fprintf(f, "ReadyForQuery\t");
-	pqTraceOutputByte1(message, &cursor, f);
+	pqTraceOutputByte1(f, message, &cursor);
 }
 
 /* RowDescription */
 static void
-pqTraceOutputT(const char *message, int end, FILE *f)
+pqTraceOutputT(FILE *f, const char *message, int end)
 {
 	int	cursor = 0;
 	int nfields;
 	int	i;
 
 	fprintf(f, "RowDescription\t");
-	nfields = pqTraceOutputInt16(message, &cursor, f);
+	nfields = pqTraceOutputInt16(f, message, &cursor);
 
 	for (i = 0; i < nfields; i++)
 	{
-		pqTraceOutputString(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
-		pqTraceOutputInt16(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
-		pqTraceOutputInt16(message, &cursor, f);
-		pqTraceOutputInt32(message, &cursor, f);
-		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputInt16(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputInt16(f, message, &cursor);
+		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputInt16(f, message, &cursor);
 	}
 }
 
@@ -569,76 +569,76 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	switch(id)
 	{
 		case 'R':	/* Authentication */
-			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputR(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'K':	/* secret key data from the backend */
-			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputK(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'B':	/* Bind */
-			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputB(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'C':	/* Close(F) or Command Complete(B) */
-			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			pqTraceOutputC(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
 			break;
 		case 'd':	/* Copy Data */
 			/* Drop COPY data to reduce the overhead of logging. */
 			break;
 		case 'f':	/* Copy Fail */
-			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputf(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'G':	/* Start Copy In */
-			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputG(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'H':	/* Flush(F) or Start Copy Out(B) */
 			if (!toServer)
-				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+				pqTraceOutputH(conn->Pfdebug, message + LogCursor, LogEnd);
 			else
 				fprintf(conn->Pfdebug, "Flush");
 			break;
 		case 'W':	/* Start Copy Both */
-			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputW(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'D':	/* Describe(F) or Data Row(B) */
-			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			pqTraceOutputD(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
 			break;
 		case 'E':	/* Execute(F) or Error Response(B) */
-			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			pqTraceOutputE(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
 			break;
 		case 'F':	/* Function Call */
-			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputF(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'V':	/* Function Call response */
-			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputV(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'v':	/* Negotiate Protocol Version */
-			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputv(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'N':	/* Notice Response */
-			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputN(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'A':	/* Notification Response */
-			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputA(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 't':	/* Parameter Description */
-			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputt(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'S':	/* Parameter Status(B) or Sync(F) */
 			if (!toServer)
-				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+				pqTraceOutputS(conn->Pfdebug, message + LogCursor, LogEnd);
 			else
 				fprintf(conn->Pfdebug, "Sync");
 			break;
 		case 'P':	/* Parse */
-			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputP(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'Q':	/* Query */
-			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputQ(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case 'Z':	/* Ready For Query */
-			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			pqTraceOutputZ(conn->Pfdebug, message + LogCursor);
 			break;
 		case 'T':	/* Row Description */
-			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			pqTraceOutputT(conn->Pfdebug, message + LogCursor, LogEnd);
 			break;
 		case '2':	/* Bind Complete */
 			fprintf(conn->Pfdebug, "BindComplete");
@@ -702,9 +702,9 @@ pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
 	{
 		case 16:	/* CancelRequest */
 			fprintf(conn->Pfdebug, "CancelRequest\t");
-			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
-			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
-			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 8 :	/* GSSENCRequest or SSLRequest */
 			/* These messages do not reach here. */
-- 
2.20.1

v30-0002-libpq_pipeline-add-t-support-for-PQtrace.patchtext/x-diff; charset=us-asciiDownload
From b32ae3805bb28553c0a1cf308c6ed27f58576f3c Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 19:12:12 -0300
Subject: [PATCH v30 2/7] libpq_pipeline: add -t support for PQtrace

---
 .../modules/libpq_pipeline/libpq_pipeline.c   | 84 +++++++++++++------
 1 file changed, 59 insertions(+), 25 deletions(-)

diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 846ee9f5ab..34d085035b 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -23,6 +23,7 @@
 #include "catalog/pg_type_d.h"
 #include "common/fe_memutils.h"
 #include "libpq-fe.h"
+#include "pg_getopt.h"
 #include "portability/instr_time.h"
 
 
@@ -30,6 +31,9 @@ static void exit_nicely(PGconn *conn);
 
 const char *const progname = "libpq_pipeline";
 
+/* Options and defaults */
+char	   *tracefile = NULL;	/* path to PQtrace() file */
+
 
 #define DEBUG
 #ifdef DEBUG
@@ -1209,8 +1213,10 @@ usage(const char *progname)
 {
 	fprintf(stderr, "%s tests libpq's pipeline mode.\n\n", progname);
 	fprintf(stderr, "Usage:\n");
-	fprintf(stderr, "  %s tests", progname);
-	fprintf(stderr, "  %s testname [conninfo [number_of_rows]]\n", progname);
+	fprintf(stderr, "  %s [OPTION] tests\n", progname);
+	fprintf(stderr, "  %s [OPTION] TESTNAME [CONNINFO [NUMBER_OF_ROWS]\n", progname);
+	fprintf(stderr, "\nOptions:\n");
+	fprintf(stderr, "  -t TRACEFILE       generate a libpq trace to TRACEFILE\n");
 }
 
 static void
@@ -1231,37 +1237,54 @@ main(int argc, char **argv)
 {
 	const char *conninfo = "";
 	PGconn	   *conn;
+	FILE	   *trace;
+	char	   *testname;
 	int			numrows = 10000;
 	PGresult   *res;
+	int			c;
 
-	if (strcmp(argv[1], "tests") == 0)
+	while ((c = getopt(argc, argv, "t:")) != -1)
 	{
-		print_test_list();
-		exit(0);
+		switch (c)
+		{
+			case 't':			/* trace file */
+				tracefile = pg_strdup(optarg);
+				break;
+		}
 	}
 
-	/*
-	 * The testname parameter is mandatory; it can be followed by a conninfo
-	 * string and number of rows.
-	 */
-	if (argc < 2 || argc > 4)
+	if (optind < argc)
+	{
+		testname = argv[optind];
+		optind++;
+	}
+	else
 	{
 		usage(argv[0]);
 		exit(1);
 	}
 
-	if (argc >= 3)
-		conninfo = pg_strdup(argv[2]);
+	if (strcmp(testname, "tests") == 0)
+	{
+		print_test_list();
+		exit(0);
+	}
 
-	if (argc >= 4)
+	if (optind < argc)
+	{
+		conninfo = argv[optind];
+		optind++;
+	}
+	if (optind < argc)
 	{
 		errno = 0;
-		numrows = strtol(argv[3], NULL, 10);
+		numrows = strtol(argv[optind], NULL, 10);
 		if (errno != 0 || numrows <= 0)
 		{
-			fprintf(stderr, "couldn't parse \"%s\" as a positive integer\n", argv[3]);
+			fprintf(stderr, "couldn't parse \"%s\" as a positive integer\n", argv[optind]);
 			exit(1);
 		}
+		optind++;
 	}
 
 	/* Make a connection to the database */
@@ -1272,30 +1295,41 @@ main(int argc, char **argv)
 				PQerrorMessage(conn));
 		exit_nicely(conn);
 	}
+
+	/* Set the trace file, if requested */
+	if (tracefile != NULL)
+	{
+		trace = fopen(tracefile, "w+");
+
+		if (trace == NULL)
+			pg_fatal("could not open file \"%s\": %m", tracefile);
+		PQtrace(conn, trace);
+		PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS);
+	}
+
 	res = PQexec(conn, "SET lc_messages TO \"C\"");
 	if (PQresultStatus(res) != PGRES_COMMAND_OK)
 		pg_fatal("failed to set lc_messages: %s", PQerrorMessage(conn));
 
-	if (strcmp(argv[1], "disallowed_in_pipeline") == 0)
+	if (strcmp(testname, "disallowed_in_pipeline") == 0)
 		test_disallowed_in_pipeline(conn);
-	else if (strcmp(argv[1], "multi_pipelines") == 0)
+	else if (strcmp(testname, "multi_pipelines") == 0)
 		test_multi_pipelines(conn);
-	else if (strcmp(argv[1], "pipeline_abort") == 0)
+	else if (strcmp(testname, "pipeline_abort") == 0)
 		test_pipeline_abort(conn);
-	else if (strcmp(argv[1], "pipelined_insert") == 0)
+	else if (strcmp(testname, "pipelined_insert") == 0)
 		test_pipelined_insert(conn, numrows);
-	else if (strcmp(argv[1], "prepared") == 0)
+	else if (strcmp(testname, "prepared") == 0)
 		test_prepared(conn);
-	else if (strcmp(argv[1], "simple_pipeline") == 0)
+	else if (strcmp(testname, "simple_pipeline") == 0)
 		test_simple_pipeline(conn);
-	else if (strcmp(argv[1], "singlerow") == 0)
+	else if (strcmp(testname, "singlerow") == 0)
 		test_singlerowmode(conn);
-	else if (strcmp(argv[1], "transaction") == 0)
+	else if (strcmp(testname, "transaction") == 0)
 		test_transaction(conn);
 	else
 	{
-		fprintf(stderr, "\"%s\" is not a recognized test name\n", argv[1]);
-		usage(argv[0]);
+		fprintf(stderr, "\"%s\" is not a recognized test name\n", testname);
 		exit(1);
 	}
 
-- 
2.20.1

v30-0001-libpq-trace-v29.patchtext/x-diff; charset=us-asciiDownload
From f239da90f069b19e1c2fa6b69bc7c6fd4be826b9 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 11:10:52 -0300
Subject: [PATCH v30 1/7] libpq trace v29

---
 doc/src/sgml/libpq.sgml             |  39 +-
 src/interfaces/libpq/Makefile       |   1 +
 src/interfaces/libpq/exports.txt    |   1 +
 src/interfaces/libpq/fe-connect.c   |  21 -
 src/interfaces/libpq/fe-exec.c      |   4 -
 src/interfaces/libpq/fe-misc.c      |  66 +--
 src/interfaces/libpq/fe-protocol3.c |   8 +
 src/interfaces/libpq/libpq-fe.h     |   2 +
 src/interfaces/libpq/libpq-int.h    |   7 +
 src/interfaces/libpq/libpq-trace.c  | 717 ++++++++++++++++++++++++++++
 10 files changed, 789 insertions(+), 77 deletions(-)
 create mode 100644 src/interfaces/libpq/libpq-trace.c

diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index be674fbaa9..838e394d54 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6459,12 +6459,27 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>&gt;</literal> for messages from client to server
+      or <literal>&lt;</literal> for messages from server to client),
+      message length, message type, and message contents.
+      Non-message contents fields (timestamp, direction, length and message type)
+      are separated by a tab. Message contents are separated by a space.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -6479,6 +6494,28 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_SUPPRESS_TIMESTAMPS</literal>,
+      then the timestamp is not included when printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882a2b..0424523492 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5c48c14191..a00701f2c5 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -183,3 +183,4 @@ PQenterPipelineMode       180
 PQexitPipelineMode        181
 PQpipelineSync            182
 PQpipelineStatus          183
+PQtraceSetFlags           184
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 53b354abb2..a90bdb8ab6 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6859,27 +6859,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index f143b8b7fb..03592bdce9 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -970,10 +970,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b91f..54d6a76246 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -84,9 +84,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +97,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +132,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +157,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +175,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +192,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +207,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +244,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +277,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +485,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +519,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +528,16 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +964,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 306e89acfd..de77c06874 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -457,6 +457,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1660,6 +1663,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2121,6 +2127,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index cee42d4843..d0decde2da 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -376,7 +376,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_SUPPRESS_TIMESTAMPS		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6374ec657a..1a1d0e1414 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -818,6 +819,12 @@ extern ssize_t pg_GSS_write(PGconn *conn, const void *ptr, size_t len);
 extern ssize_t pg_GSS_read(PGconn *conn, void *ptr, size_t len);
 #endif
 
+/* === in libpq-trace.c === */
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message,
+															bool toServer);
+extern void pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message);
+
 /* === miscellaneous macros === */
 
 /*
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000000..bacb7903b9
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,717 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(const char *data, int *cursor, FILE *pfdebug)
+{
+	const char *v = data + *cursor;
+
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*v))
+		fprintf(pfdebug, " \\x%02x", *v);
+	else
+		fprintf(pfdebug, " %c", *v);
+	++*cursor;
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(const char *data, int *cursor, FILE *pfdebug)
+{
+	uint16		tmp;
+	int			result;
+
+	memcpy(&tmp, data + *cursor , 2);
+	*cursor += 2;
+	result = (int) pg_ntoh16(tmp);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(const char *data, int *cursor, FILE *pfdebug)
+{
+	int			result;
+
+	memcpy(&result, data + *cursor, 4);
+	*cursor += 4;
+	result = (int) pg_ntoh32(result);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(const char *data, int *cursor, FILE *pfdebug)
+{
+	int	len;
+
+	len = fprintf(pfdebug, " \"%s\"", data + *cursor);
+
+	/*
+	 * This is null-terminated string. So add 1 after subtracting 3
+	 * which is the double quotes and space length from len.
+	 */
+	*cursor += (len - 3 + 1);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(const char *data, int *cursor, FILE *pfdebug, int len)
+{
+	int			i,
+				next;			/* first char not yet printed */
+	const char	*v = data + *cursor;
+
+	fprintf(pfdebug, " \'");
+
+	for (next = i = 0; i < len; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < len)
+		fwrite(v + next, 1, len - next, pfdebug);
+
+	fprintf(pfdebug, "\'");
+	*cursor += len;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "Authentication\t");
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "BackendKeyData\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(const char *message, int end, FILE *f)
+{
+	int cursor = 0;
+	int nparams;
+	int nbytes;
+	int i;
+
+	fprintf(f, "Bind\t");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "Close\t");
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, f);
+	}
+	else
+	{
+		fprintf(f, "CommandComplete\t");
+		pqTraceOutputString(message, &cursor, f);
+	}
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "CopyFail\t");
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "CopyInResponse\t");
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "CopyOutResponse\t");
+	pqTraceOutputByte1(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "CopyBothResponse\t");
+	pqTraceOutputByte1(message, &cursor, f);
+
+	while (end > cursor)
+		pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "Describe\t");
+		pqTraceOutputByte1(message, &cursor, f);
+		pqTraceOutputString(message, &cursor, f);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		fprintf(f, "DataRow\t");
+		nfields = pqTraceOutputInt16(message, &cursor, f);
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(message, &cursor, f);
+			if (len == -1)
+				continue;
+			pqTraceOutputNchar(message, &cursor, f, len);
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(const char *message, int end, FILE *f, bool toServer)
+{
+	int	cursor = 0;
+
+	if (toServer)
+	{
+		fprintf(f, "Execute\t");
+		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+	}
+	else
+	{
+		fprintf(f, "ErrorResponse\t");
+		while (end > cursor)
+		{
+			pqTraceOutputByte1(message, &cursor, f);
+			if (message[cursor] == '\0')
+				continue;
+			pqTraceOutputString(message, &cursor, f);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int nbytes;
+	int	i;
+
+	fprintf(f, "FunctionCall\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt16(message, &cursor, f);
+
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		nbytes = pqTraceOutputInt32(message, &cursor, f);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(message, &cursor, f, nbytes);
+	}
+
+	pqTraceOutputInt16(message, &cursor, f);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int		len;
+
+	fprintf(f, "FunctionCallResponse\t");
+	len = pqTraceOutputInt32(message, &cursor, f);
+	if (len != -1)
+		pqTraceOutputNchar(message, &cursor, f, len);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "NegotiateProtocolVersion\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "NoticeResponse\t");
+	while (end > cursor)
+	{
+		pqTraceOutputByte1(message, &cursor, f);
+		if (message[cursor] == '\0')
+			continue;
+		pqTraceOutputString(message, &cursor, f);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "NotificationResponse\t");
+	pqTraceOutputInt32(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(const char *message, FILE *f)
+{
+	int	cursor = 0;
+	int	nfields;
+	int	i;
+
+	fprintf(f, "ParameterDescription\t");
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "ParameterStatus\t");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* Parse */
+static void
+pqTraceOutputP(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nparams;
+	int i;
+
+	fprintf(f, "Parse\t");
+	pqTraceOutputString(message, &cursor, f);
+	pqTraceOutputString(message, &cursor, f);
+	nparams = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nparams; i++)
+		pqTraceOutputInt32(message, &cursor, f);
+}
+
+/* Query */
+static void
+pqTraceOutputQ(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "Query\t");
+	pqTraceOutputString(message, &cursor, f);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(const char *message, FILE *f)
+{
+	int	cursor = 0;
+
+	fprintf(f, "ReadyForQuery\t");
+	pqTraceOutputByte1(message, &cursor, f);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(const char *message, int end, FILE *f)
+{
+	int	cursor = 0;
+	int nfields;
+	int	i;
+
+	fprintf(f, "RowDescription\t");
+	nfields = pqTraceOutputInt16(message, &cursor, f);
+
+	for (i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+		pqTraceOutputInt32(message, &cursor, f);
+		pqTraceOutputInt16(message, &cursor, f);
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	char		timestr[128];
+	char 		id;
+	int			length;
+	char	   *prefix = toServer ? ">" : "<";
+	int			LogCursor = 0;
+	int			LogEnd;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+	LogEnd = length - 4;
+
+	fprintf(conn->Pfdebug, "%s\t%s\t%d\t", timestr, prefix, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (!toServer)
+				pqTraceOutputH(message + LogCursor, LogEnd, conn->Pfdebug);
+			else
+				fprintf(conn->Pfdebug, "Flush");
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'E':	/* Execute(F) or Error Response(B) */
+			pqTraceOutputE(message + LogCursor, LogEnd, conn->Pfdebug, toServer);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (!toServer)
+				pqTraceOutputS(message + LogCursor, LogEnd, conn->Pfdebug);
+			else
+				fprintf(conn->Pfdebug, "Sync");
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(message + LogCursor, conn->Pfdebug);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(message + LogCursor, LogEnd, conn->Pfdebug);
+			break;
+		case '2':	/* Bind Complete */
+			fprintf(conn->Pfdebug, "BindComplete");
+			/* No message content */
+			break;
+		case '3':	/* Close Complete */
+			fprintf(conn->Pfdebug, "CloseComplete");
+			/* No message content */
+			break;
+		case 'c':	/* Copy Done */
+			fprintf(conn->Pfdebug, "CopyDone");
+			/* No message content */
+			break;
+		case 'I':	/* Empty Query Response */
+			fprintf(conn->Pfdebug, "EmptyQueryResponse");
+			/* No message content */
+			break;
+		case 'n':	/* No Data */
+			fprintf(conn->Pfdebug, "NoData");
+			/* No message content */
+			break;
+		case '1':	/* Parse Complete */
+			fprintf(conn->Pfdebug, "ParseComplete");
+			/* No message content */
+			break;
+		case 's':	/* Portal Suspended */
+			fprintf(conn->Pfdebug, "PortalSuspended");
+			/* No message content */
+			break;
+		case 'X':	/* Terminate */
+			fprintf(conn->Pfdebug, "Terminate");
+			/* No message content */
+			break;
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: %02x", id);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
+
+void
+pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
+{
+	char		timestr[128];
+	int			length;
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	fprintf(conn->Pfdebug, "%s\t>\t%d\t", timestr, length);
+
+	switch(length)
+	{
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "CancelRequest\t");
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			pqTraceOutputInt32(message, &LogCursor, conn->Pfdebug);
+			break;
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messages do not reach here. */
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: length is %d", length);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
-- 
2.20.1

v30-0004-use-F-B-instead-of.patchtext/x-diff; charset=us-asciiDownload
From d44009a25de3af549dfa528a4fa44c9ebf785c42 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 19:13:16 -0300
Subject: [PATCH v30 4/7] use F/B instead of </>

---
 src/interfaces/libpq/libpq-trace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index bff542cada..94f28f04a8 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -548,7 +548,7 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	char		timestr[128];
 	char 		id;
 	int			length;
-	char	   *prefix = toServer ? ">" : "<";
+	char	   *prefix = toServer ? "F" : "B";
 	int			LogCursor = 0;
 	int			LogEnd;
 
-- 
2.20.1

v30-0005-pass-cursor-to-printing-routines.patchtext/x-diff; charset=us-asciiDownload
From 39a5e0c6b5559a0fb9a6c376e493e4ee4bc980fb Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 20:04:53 -0300
Subject: [PATCH v30 5/7] pass *cursor to printing routines

---
 src/interfaces/libpq/libpq-trace.c | 309 +++++++++++++----------------
 1 file changed, 134 insertions(+), 175 deletions(-)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index 94f28f04a8..06d8d981a3 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -106,7 +106,7 @@ pqTraceOutputByte1(FILE *pfdebug, const char *data, int *cursor)
 		fprintf(pfdebug, " \\x%02x", *v);
 	else
 		fprintf(pfdebug, " %c", *v);
-	++*cursor;
+	*cursor += 1;
 }
 
 /*
@@ -195,142 +195,125 @@ pqTraceOutputNchar(FILE *pfdebug, int len, const char *data, int *cursor)
 
 /* Authentication */
 static void
-pqTraceOutputR(FILE *f, const char *message)
+pqTraceOutputR(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "Authentication\t");
-	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
 }
 
 /* BackendKeyData */
 static void
-pqTraceOutputK(FILE *f, const char *message)
+pqTraceOutputK(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "BackendKeyData\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputInt32(f, message, cursor);
 }
 
 /* Bind */
 static void
-pqTraceOutputB(FILE *f, const char *message, int end)
+pqTraceOutputB(FILE *f, const char *message, int *cursor)
 {
-	int cursor = 0;
 	int nparams;
-	int nbytes;
-	int i;
 
 	fprintf(f, "Bind\t");
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
-	nparams = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt16(f, message, cursor);
 
-	nparams = pqTraceOutputInt16(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nparams; i++)
+	for (int i = 0; i < nparams; i++)
 	{
-		nbytes = pqTraceOutputInt32(f, message, &cursor);
+		int nbytes;
+
+		nbytes = pqTraceOutputInt32(f, message, cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(f, nbytes, message, &cursor);
+		pqTraceOutputNchar(f, nbytes, message, cursor);
 	}
 
-	nparams = pqTraceOutputInt16(f, message, &cursor);
-	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* Close(F) or CommandComplete(B) */
 static void
-pqTraceOutputC(FILE *f, bool toServer, const char *message, int end)
+pqTraceOutputC(FILE *f, bool toServer, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	if (toServer)
 	{
 		fprintf(f, "Close\t");
-		pqTraceOutputByte1(f, message, &cursor);
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputByte1(f, message, cursor);
+		pqTraceOutputString(f, message, cursor);
 	}
 	else
 	{
 		fprintf(f, "CommandComplete\t");
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputString(f, message, cursor);
 	}
 }
 
 /* CopyFail */
 static void
-pqTraceOutputf(FILE *f, const char *message, int end)
+pqTraceOutputf(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "CopyFail\t");
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* CopyInResponse */
 static void
-pqTraceOutputG(FILE *f, const char *message, int end)
+pqTraceOutputG(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int	nfields;
-	int	i;
 
 	fprintf(f, "CopyInResponse\t");
-	pqTraceOutputByte1(f, message, &cursor);
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* CopyOutResponse */
 static void
-pqTraceOutputH(FILE *f, const char *message, int end)
+pqTraceOutputH(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int	nfields;
-	int	i;
 
 	fprintf(f, "CopyOutResponse\t");
-	pqTraceOutputByte1(f, message, &cursor);
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* CopyBothResponse */
 static void
-pqTraceOutputW(FILE *f, const char *message, int end)
+pqTraceOutputW(FILE *f, const char *message, int *cursor, int length)
 {
-	int	cursor = 0;
-
 	fprintf(f, "CopyBothResponse\t");
-	pqTraceOutputByte1(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
 
-	while (end > cursor)
-		pqTraceOutputInt16(f, message, &cursor);
+	while (length > *cursor)
+		pqTraceOutputInt16(f, message, cursor);
 }
 
 /* Describe(F) or DataRow(B) */
 static void
-pqTraceOutputD(FILE *f, bool toServer, const char *message, int end)
+pqTraceOutputD(FILE *f, bool toServer, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	if (toServer)
 	{
 		fprintf(f, "Describe\t");
-		pqTraceOutputByte1(f, message, &cursor);
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputByte1(f, message, cursor);
+		pqTraceOutputString(f, message, cursor);
 	}
 	else
 	{
@@ -339,206 +322,183 @@ pqTraceOutputD(FILE *f, bool toServer, const char *message, int end)
 		int		i;
 
 		fprintf(f, "DataRow\t");
-		nfields = pqTraceOutputInt16(f, message, &cursor);
+		nfields = pqTraceOutputInt16(f, message, cursor);
 		for (i = 0; i < nfields; i++)
 		{
-			len = pqTraceOutputInt32(f, message, &cursor);
+			len = pqTraceOutputInt32(f, message, cursor);
 			if (len == -1)
 				continue;
-			pqTraceOutputNchar(f, len, message, &cursor);
+			pqTraceOutputNchar(f, len, message, cursor);
 		}
 	}
 }
 
 /* Execute(F) or ErrorResponse(B) */
 static void
-pqTraceOutputE(FILE *f, bool toServer, const char *message, int end)
+pqTraceOutputE(FILE *f, bool toServer, const char *message, int *cursor, int length)
 {
-	int	cursor = 0;
-
 	if (toServer)
 	{
 		fprintf(f, "Execute\t");
-		pqTraceOutputString(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
+		pqTraceOutputString(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
 	}
 	else
 	{
 		fprintf(f, "ErrorResponse\t");
-		while (end > cursor)
+		for (;;)
 		{
-			pqTraceOutputByte1(f, message, &cursor);
-			if (message[cursor] == '\0')
-				continue;
-			pqTraceOutputString(f, message, &cursor);
+			pqTraceOutputByte1(f, message, cursor);
+			if (message[*cursor - 1] == '\0')
+				break;
+			pqTraceOutputString(f, message, cursor);
 		}
 	}
 }
 
 /* FunctionCall */
 static void
-pqTraceOutputF(FILE *f, const char *message)
+pqTraceOutputF(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int nfields;
 	int nbytes;
-	int	i;
 
 	fprintf(f, "FunctionCall\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt16(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
 
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
+	for (int i = 0; i < nfields; i++)
 	{
-		nbytes = pqTraceOutputInt32(f, message, &cursor);
+		nbytes = pqTraceOutputInt32(f, message, cursor);
 		if (nbytes == -1)
 			continue;
-		pqTraceOutputNchar(f, nbytes, message, &cursor);
+		pqTraceOutputNchar(f, nbytes, message, cursor);
 	}
 
-	pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputInt16(f, message, cursor);
 }
 
 /* FunctionCallResponse */
 static void
-pqTraceOutputV(FILE *f, const char *message)
+pqTraceOutputV(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int		len;
 
 	fprintf(f, "FunctionCallResponse\t");
-	len = pqTraceOutputInt32(f, message, &cursor);
+	len = pqTraceOutputInt32(f, message, cursor);
 	if (len != -1)
-		pqTraceOutputNchar(f, len, message, &cursor);
+		pqTraceOutputNchar(f, len, message, cursor);
 }
 
 /* NegotiateProtocolVersion */
 static void
-pqTraceOutputv(FILE *f, const char *message)
+pqTraceOutputv(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "NegotiateProtocolVersion\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	pqTraceOutputInt32(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputInt32(f, message, cursor);
 }
 
 /* NoticeResponse */
 static void
-pqTraceOutputN(FILE *f, const char *message, int end)
+pqTraceOutputN(FILE *f, const char *message, int *cursor, int length)
 {
-	int	cursor = 0;
-
 	fprintf(f, "NoticeResponse\t");
-	while (end > cursor)
+	for (;;)
 	{
-		pqTraceOutputByte1(f, message, &cursor);
-		if (message[cursor] == '\0')
-			continue;
-		pqTraceOutputString(f, message, &cursor);
+		pqTraceOutputByte1(f, message, cursor);
+		if (message[*cursor - 1] == '\0')
+			break;
+		pqTraceOutputString(f, message, cursor);
 	}
 }
 
 /* NotificationResponse */
 static void
-pqTraceOutputA(FILE *f, const char *message, int end)
+pqTraceOutputA(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "NotificationResponse\t");
-	pqTraceOutputInt32(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* ParameterDescription */
 static void
-pqTraceOutputt(FILE *f, const char *message)
+pqTraceOutputt(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int	nfields;
-	int	i;
 
 	fprintf(f, "ParameterDescription\t");
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
-		pqTraceOutputInt32(f, message, &cursor);
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt32(f, message, cursor);
 }
 
 /* ParameterStatus */
 static void
-pqTraceOutputS(FILE *f, const char *message, int end)
+pqTraceOutputS(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "ParameterStatus\t");
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* Parse */
 static void
-pqTraceOutputP(FILE *f, const char *message, int end)
+pqTraceOutputP(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int nparams;
-	int i;
 
 	fprintf(f, "Parse\t");
-	pqTraceOutputString(f, message, &cursor);
-	pqTraceOutputString(f, message, &cursor);
-	nparams = pqTraceOutputInt16(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nparams; i++)
-		pqTraceOutputInt32(f, message, &cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt32(f, message, cursor);
 }
 
 /* Query */
 static void
-pqTraceOutputQ(FILE *f, const char *message, int end)
+pqTraceOutputQ(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "Query\t");
-	pqTraceOutputString(f, message, &cursor);
+	pqTraceOutputString(f, message, cursor);
 }
 
 /* ReadyForQuery */
 static void
-pqTraceOutputZ(FILE *f, const char *message)
+pqTraceOutputZ(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
-
 	fprintf(f, "ReadyForQuery\t");
-	pqTraceOutputByte1(f, message, &cursor);
+	pqTraceOutputByte1(f, message, cursor);
 }
 
 /* RowDescription */
 static void
-pqTraceOutputT(FILE *f, const char *message, int end)
+pqTraceOutputT(FILE *f, const char *message, int *cursor)
 {
-	int	cursor = 0;
 	int nfields;
-	int	i;
 
 	fprintf(f, "RowDescription\t");
-	nfields = pqTraceOutputInt16(f, message, &cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
 
-	for (i = 0; i < nfields; i++)
+	for (int i = 0; i < nfields; i++)
 	{
-		pqTraceOutputString(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
-		pqTraceOutputInt16(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
-		pqTraceOutputInt16(f, message, &cursor);
-		pqTraceOutputInt32(f, message, &cursor);
-		pqTraceOutputInt16(f, message, &cursor);
+		pqTraceOutputString(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
 	}
 }
 
@@ -550,95 +510,94 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	int			length;
 	char	   *prefix = toServer ? "F" : "B";
 	int			LogCursor = 0;
-	int			LogEnd;
 
 	if ((conn->traceFlags & PQTRACE_SUPPRESS_TIMESTAMPS) == 0)
+	{
 		pqTraceFormatTimestamp(timestr, sizeof(timestr));
-	else
-		timestr[0] = '\0';
+		fprintf(conn->Pfdebug, "%s\t", timestr);
+	}
 
 	id = message[LogCursor++];
 
-	memcpy(&length, message + LogCursor , 4);
+	memcpy(&length, message + LogCursor, 4);
 	length = (int) pg_ntoh32(length);
 	LogCursor += 4;
-	LogEnd = length - 4;
 
-	fprintf(conn->Pfdebug, "%s\t%s\t%d\t", timestr, prefix, length);
+	fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
 
 	switch(id)
 	{
 		case 'R':	/* Authentication */
-			pqTraceOutputR(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputR(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'K':	/* secret key data from the backend */
-			pqTraceOutputK(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputK(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'B':	/* Bind */
-			pqTraceOutputB(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputB(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'C':	/* Close(F) or Command Complete(B) */
-			pqTraceOutputC(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
+			pqTraceOutputC(conn->Pfdebug, toServer, message, &LogCursor);
 			break;
 		case 'd':	/* Copy Data */
 			/* Drop COPY data to reduce the overhead of logging. */
 			break;
 		case 'f':	/* Copy Fail */
-			pqTraceOutputf(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputf(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'G':	/* Start Copy In */
-			pqTraceOutputG(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputG(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'H':	/* Flush(F) or Start Copy Out(B) */
 			if (!toServer)
-				pqTraceOutputH(conn->Pfdebug, message + LogCursor, LogEnd);
+				pqTraceOutputH(conn->Pfdebug, message, &LogCursor);
 			else
 				fprintf(conn->Pfdebug, "Flush");
 			break;
 		case 'W':	/* Start Copy Both */
-			pqTraceOutputW(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputW(conn->Pfdebug, message, &LogCursor, length);
 			break;
 		case 'D':	/* Describe(F) or Data Row(B) */
-			pqTraceOutputD(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
+			pqTraceOutputD(conn->Pfdebug, toServer, message, &LogCursor);
 			break;
 		case 'E':	/* Execute(F) or Error Response(B) */
-			pqTraceOutputE(conn->Pfdebug, toServer, message + LogCursor, LogEnd);
+			pqTraceOutputE(conn->Pfdebug, toServer, message, &LogCursor, length);
 			break;
 		case 'F':	/* Function Call */
-			pqTraceOutputF(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputF(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'V':	/* Function Call response */
-			pqTraceOutputV(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputV(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'v':	/* Negotiate Protocol Version */
-			pqTraceOutputv(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputv(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'N':	/* Notice Response */
-			pqTraceOutputN(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputN(conn->Pfdebug, message, &LogCursor, length);
 			break;
 		case 'A':	/* Notification Response */
-			pqTraceOutputA(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputA(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 't':	/* Parameter Description */
-			pqTraceOutputt(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputt(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'S':	/* Parameter Status(B) or Sync(F) */
 			if (!toServer)
-				pqTraceOutputS(conn->Pfdebug, message + LogCursor, LogEnd);
+				pqTraceOutputS(conn->Pfdebug, message, &LogCursor);
 			else
 				fprintf(conn->Pfdebug, "Sync");
 			break;
 		case 'P':	/* Parse */
-			pqTraceOutputP(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputP(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'Q':	/* Query */
-			pqTraceOutputQ(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputQ(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'Z':	/* Ready For Query */
-			pqTraceOutputZ(conn->Pfdebug, message + LogCursor);
+			pqTraceOutputZ(conn->Pfdebug, message, &LogCursor);
 			break;
 		case 'T':	/* Row Description */
-			pqTraceOutputT(conn->Pfdebug, message + LogCursor, LogEnd);
+			pqTraceOutputT(conn->Pfdebug, message, &LogCursor);
 			break;
 		case '2':	/* Bind Complete */
 			fprintf(conn->Pfdebug, "BindComplete");
-- 
2.20.1

v30-0006-add-length-check.patchtext/x-diff; charset=us-asciiDownload
From 4d645bdf08de419b207b0064986d81f758900809 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 26 Mar 2021 20:05:10 -0300
Subject: [PATCH v30 6/7] add length check

---
 src/interfaces/libpq/libpq-trace.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
index 06d8d981a3..6c58a578ac 100644
--- a/src/interfaces/libpq/libpq-trace.c
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -637,6 +637,16 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	}
 
 	fputc('\n', conn->Pfdebug);
+
+	/*
+	 * Verify the printing routine did it right.  Note that the one-byte
+	 * message identifier is not included in the length, but our cursor
+	 * does include it.
+	 */
+	if (LogCursor - 1 != length)
+		fprintf(conn->Pfdebug,
+				"mismatched message length: consumed %d, expected %d\n",
+				LogCursor - 1, length);
 }
 
 void
-- 
2.20.1

v30-0007-Use-libpq_pipeline-to-test-PQtrace.patchtext/x-diff; charset=us-asciiDownload
From d040e90b1ed206a14b031ac92f494b87b1d32ae8 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Sat, 27 Mar 2021 16:10:08 -0300
Subject: [PATCH v30 7/7] Use libpq_pipeline to test PQtrace

---
 .../libpq_pipeline/t/001_libpq_pipeline.pl    | 24 ++++++++++-
 .../traces/multi_pipelines.trace              | 26 ++++++++++++
 .../libpq_pipeline/traces/prepared.trace      | 21 ++++++++++
 .../traces/simple_pipeline.trace              | 15 +++++++
 .../libpq_pipeline/traces/singlerow.trace     | 42 +++++++++++++++++++
 5 files changed, 126 insertions(+), 2 deletions(-)
 create mode 100644 src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
 create mode 100644 src/test/modules/libpq_pipeline/traces/prepared.trace
 create mode 100644 src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
 create mode 100644 src/test/modules/libpq_pipeline/traces/singlerow.trace

diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0213f21ee8..a0ee8640a7 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -4,7 +4,7 @@ use warnings;
 use Config;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 8;
+use Test::More tests => 12;
 use Cwd;
 
 my $node = get_new_node('main');
@@ -20,9 +20,29 @@ my @tests = split(/\s+/, $out);
 
 for my $testname (@tests)
 {
+	my @extraargs = ();
+	my $cmptrace = grep(/^$testname$/,
+		qw(simple_pipeline multi_pipelines prepared singlerow)) > 0;
+	my $traceout = "$TestLib::log_path/$testname.trace";
+
+	# For a bunch of tests, generate a libpq trace file too.
+	if ($cmptrace)
+	{
+		push @extraargs, "-t", $traceout;
+	}
+
+
 	$node->command_ok(
-		[ 'libpq_pipeline', $testname, $node->connstr('postgres'), $numrows ],
+		[ 'libpq_pipeline', @extraargs, $testname, $node->connstr('postgres'), $numrows ],
 		"libpq_pipeline $testname");
+
+	if ($cmptrace)
+	{
+		my $expected = slurp_file("traces/$testname.trace");
+		my $result = slurp_file($traceout);
+
+		is($expected, $result, "$testname trace match");
+	}
 }
 
 $node->stop('fast');
diff --git a/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace b/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
new file mode 100644
index 0000000000..f7dfbef11f
--- /dev/null
+++ b/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
@@ -0,0 +1,26 @@
+F	27	Query	 "SET lc_messages TO "C""
+B	8	CommandComplete	 "SET"
+B	5	ReadyForQuery	 I
+F	21	Parse	 "" "SELECT $1" 1 23
+F	19	Bind	 "" "" 0 1 1 '1' 1 0
+F	6	Describe	 P ""
+F	9	Execute	 "" 0
+F	4	Sync
+F	21	Parse	 "" "SELECT $1" 1 23
+F	19	Bind	 "" "" 0 1 1 '1' 1 0
+F	6	Describe	 P ""
+F	9	Execute	 "" 0
+F	4	Sync
+B	4	ParseComplete
+B	4	BindComplete
+B	33	RowDescription	 1 "?column?" 0 0 23 4 -1 0
+B	11	DataRow	 1 1 '1'
+B	13	CommandComplete	 "SELECT 1"
+B	5	ReadyForQuery	 I
+B	4	ParseComplete
+B	4	BindComplete
+B	33	RowDescription	 1 "?column?" 0 0 23 4 -1 0
+B	11	DataRow	 1 1 '1'
+B	13	CommandComplete	 "SELECT 1"
+B	5	ReadyForQuery	 I
+F	4	Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/prepared.trace b/src/test/modules/libpq_pipeline/traces/prepared.trace
new file mode 100644
index 0000000000..62c5c38b6a
--- /dev/null
+++ b/src/test/modules/libpq_pipeline/traces/prepared.trace
@@ -0,0 +1,21 @@
+F	27	Query	 "SET lc_messages TO "C""
+B	8	CommandComplete	 "SET"
+B	5	ReadyForQuery	 I
+F	68	Parse	 "select_one" "SELECT $1, '42', $1::numeric, interval '1 sec'" 1 23
+F	16	Describe	 S "select_one"
+F	4	Sync
+B	4	ParseComplete
+B	10	ParameterDescription	 1 23
+B	113	RowDescription	 4 "?column?" 0 0 23 4 -1 0 "?column?" 0 0 25 65535 -1 0 "numeric" 0 0 1700 65535 -1 0 "interval" 0 0 1186 16 -1 0
+B	5	ReadyForQuery	 I
+F	10	Query	 "BEGIN"
+B	10	CommandComplete	 "BEGIN"
+B	5	ReadyForQuery	 T
+F	43	Query	 "DECLARE cursor_one CURSOR FOR SELECT 1"
+B	19	CommandComplete	 "DECLARE CURSOR"
+B	5	ReadyForQuery	 T
+F	16	Describe	 P "cursor_one"
+F	4	Sync
+B	33	RowDescription	 1 "?column?" 0 0 23 4 -1 0
+B	5	ReadyForQuery	 T
+F	4	Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace b/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
new file mode 100644
index 0000000000..82a80c9191
--- /dev/null
+++ b/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
@@ -0,0 +1,15 @@
+F	27	Query	 "SET lc_messages TO "C""
+B	8	CommandComplete	 "SET"
+B	5	ReadyForQuery	 I
+F	21	Parse	 "" "SELECT $1" 1 23
+F	19	Bind	 "" "" 0 1 1 '1' 1 0
+F	6	Describe	 P ""
+F	9	Execute	 "" 0
+F	4	Sync
+B	4	ParseComplete
+B	4	BindComplete
+B	33	RowDescription	 1 "?column?" 0 0 23 4 -1 0
+B	11	DataRow	 1 1 '1'
+B	13	CommandComplete	 "SELECT 1"
+B	5	ReadyForQuery	 I
+F	4	Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/singlerow.trace b/src/test/modules/libpq_pipeline/traces/singlerow.trace
new file mode 100644
index 0000000000..7708569f39
--- /dev/null
+++ b/src/test/modules/libpq_pipeline/traces/singlerow.trace
@@ -0,0 +1,42 @@
+F	27	Query	 "SET lc_messages TO "C""
+B	8	CommandComplete	 "SET"
+B	5	ReadyForQuery	 I
+F	38	Parse	 "" "SELECT generate_series(42, $1)" 0
+F	20	Bind	 "" "" 0 1 2 '44' 1 0
+F	6	Describe	 P ""
+F	9	Execute	 "" 0
+F	38	Parse	 "" "SELECT generate_series(42, $1)" 0
+F	20	Bind	 "" "" 0 1 2 '45' 1 0
+F	6	Describe	 P ""
+F	9	Execute	 "" 0
+F	38	Parse	 "" "SELECT generate_series(42, $1)" 0
+F	20	Bind	 "" "" 0 1 2 '46' 1 0
+F	6	Describe	 P ""
+F	9	Execute	 "" 0
+F	4	Sync
+B	4	ParseComplete
+B	4	BindComplete
+B	40	RowDescription	 1 "generate_series" 0 0 23 4 -1 0
+B	12	DataRow	 1 2 '42'
+B	12	DataRow	 1 2 '43'
+B	12	DataRow	 1 2 '44'
+B	13	CommandComplete	 "SELECT 3"
+B	4	ParseComplete
+B	4	BindComplete
+B	40	RowDescription	 1 "generate_series" 0 0 23 4 -1 0
+B	12	DataRow	 1 2 '42'
+B	12	DataRow	 1 2 '43'
+B	12	DataRow	 1 2 '44'
+B	12	DataRow	 1 2 '45'
+B	13	CommandComplete	 "SELECT 4"
+B	4	ParseComplete
+B	4	BindComplete
+B	40	RowDescription	 1 "generate_series" 0 0 23 4 -1 0
+B	12	DataRow	 1 2 '42'
+B	12	DataRow	 1 2 '43'
+B	12	DataRow	 1 2 '44'
+B	12	DataRow	 1 2 '45'
+B	12	DataRow	 1 2 '46'
+B	13	CommandComplete	 "SELECT 5"
+B	5	ReadyForQuery	 I
+F	4	Terminate
-- 
2.20.1

#184alvherre@alvh.no-ip.org
alvherre@alvh.no-ip.org
In reply to: alvherre@alvh.no-ip.org (#183)
Re: libpq debug log

On 2021-Mar-27, alvherre@alvh.no-ip.org wrote:

This last one uses libpq_pipeline -t and verifies the output against an
expected trace file. Applies on top of all the previous patches. I
attach the whole lot, so that the CF bot has a chance to run it.

All tests pass, but CFbot does not run src/test/modules, so it's not
saying much.

--
�lvaro Herrera 39�49'30"S 73�17'W

#185tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: alvherre@alvh.no-ip.org (#181)
RE: libpq debug log

From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>

I added an option to the new libpq_pipeline program that it activates
libpq trace. It works nicely and I think we can add that to the
regression tests.

That's good news. Thank you.

1. The trace output for the error message won't be very nice, because it
prints line numbers. So if I want to match the output to an "expected"
file, it would break every time somebody edits the source file where the
error originates and the ereport() line is moved. For example:

(Hey, what the heck is that "Z" at the end of the message? I thought
the error ended right at the \x00 ...)

We'll investigate these issues.

2. The < and > characters are not good for visual inspection. I
replaced them with F and B and I think it's much clearer. Compare:
I think the second one is much easier on the eye.

Yes, agreed. I too thought of something like "C->S" and "S->C" because client and server should be more familiar for users than frontend and backend.

Regards
Takayuki Tsunakawa

#186tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: alvherre@alvh.no-ip.org (#183)
RE: libpq debug log

From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>

Proposed changes on top of v29.

This last one uses libpq_pipeline -t and verifies the output against an expected
trace file. Applies on top of all the previous patches. I attach the whole lot,
so that the CF bot has a chance to run it.

Thank you for polishing the patch.

Iwata-san,
Please review Alvaro-san's code, and I think you can integrate all patches into one except for 0002 and 0007. Those two patches may be separate or merged into one as a test patch.

I did notice another problem for comparison of expected trace files, which is
that RowDescription includes table OIDs for some columns. I think we would
need to have a flag to suppress that too, somehow, although the answer to what
should we do is not as clear as for the other two cases.

I'm afraid this may render the test comparison almost impossible. Tests that access system catalogs and large objects probably output OIDs.

Regards
Takayuki Tsunakawa

#187'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: tsunakawa.takay@fujitsu.com (#185)
Re: libpq debug log

On 2021-Mar-29, tsunakawa.takay@fujitsu.com wrote:

(Hey, what the heck is that "Z" at the end of the message? I thought
the error ended right at the \x00 ...)

We'll investigate these issues.

For what it's worth, I did fix this problem in patch 0005 that I
attached. The problem was that one "continue" should have been "break",
and also a "while ( .. )" needed to be made an infinite loop. It was
easy to catch these problems once I added (in 0006) the check that the
bytes consumed equal message length, as I had suggested a couple of
weeks ago :-) I also changed the code for Notice, but I didn't actually
verify that one.

2. The < and > characters are not good for visual inspection. I
replaced them with F and B and I think it's much clearer. Compare:
I think the second one is much easier on the eye.

Yes, agreed. I too thought of something like "C->S" and "S->C"
because client and server should be more familiar for users than
frontend and backend.

Hmm, yeah, that's a reasonable option too. What do others think?

--
�lvaro Herrera Valdivia, Chile

#188Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: 'alvherre@alvh.no-ip.org' (#187)
Re: libpq debug log

At Mon, 29 Mar 2021 00:02:58 -0300, "'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> wrote in

On 2021-Mar-29, tsunakawa.takay@fujitsu.com wrote:

(Hey, what the heck is that "Z" at the end of the message? I thought
the error ended right at the \x00 ...)

We'll investigate these issues.

For what it's worth, I did fix this problem in patch 0005 that I
attached. The problem was that one "continue" should have been "break",
and also a "while ( .. )" needed to be made an infinite loop. It was
easy to catch these problems once I added (in 0006) the check that the
bytes consumed equal message length, as I had suggested a couple of
weeks ago :-) I also changed the code for Notice, but I didn't actually
verify that one.

2. The < and > characters are not good for visual inspection. I
replaced them with F and B and I think it's much clearer. Compare:
I think the second one is much easier on the eye.

Yes, agreed. I too thought of something like "C->S" and "S->C"
because client and server should be more familiar for users than
frontend and backend.

Hmm, yeah, that's a reasonable option too. What do others think?

It's better to be short as far as it is clear enough. Actually '<' to
'F' and '>' to 'B' is clear enough to me. So I don't need a longer
notation. O(ut) and (I)n also makes sense to me. Rather, "C->S", and
"S->C" are a little difficult to understand at a glance

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#189tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: Kyotaro Horiguchi (#188)
RE: libpq debug log

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>

It's better to be short as far as it is clear enough. Actually '<' to
'F' and '>' to 'B' is clear enough to me. So I don't need a longer
notation.

Agreed, because the message format description in the PG manual uses F and B.

Regards
Takayuki Tsunakawa

#190tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: 'alvherre@alvh.no-ip.org' (#187)
RE: libpq debug log

From: 'alvherre@alvh.no-ip.org' <alvherre@alvh.no-ip.org>

(Hey, what the heck is that "Z" at the end of the message? I thought
the error ended right at the \x00 ...)

We'll investigate these issues.

For what it's worth, I did fix this problem in patch 0005 that I
attached. The problem was that one "continue" should have been "break",
and also a "while ( .. )" needed to be made an infinite loop. It was
easy to catch these problems once I added (in 0006) the check that the
bytes consumed equal message length, as I had suggested a couple of
weeks ago :-) I also changed the code for Notice, but I didn't actually
verify that one.

You already fix the issue, didn't you? Thank you. Your suggestion of checking the length has surely proved to be correct.

Regards
Takayuki Tsunakawa

#191iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: tsunakawa.takay@fujitsu.com (#186)
1 attachment(s)
RE: libpq debug log

Hi Alvaro san, Tsunakawa san

Thank you for creating the v30 patch.

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa.takay@fujitsu.com>
Sent: Monday, March 29, 2021 9:45 AM

...

Iwata-san,
Please review Alvaro-san's code, and I think you can integrate all patches into
one except for 0002 and 0007. Those two patches may be separate or
merged into one as a test patch.

I reviewed v30 patches. I think it was good except that the documentation about the direction of the message was not changing.
I also tried the v30 patch using regression test and it worked fine.
I merged v30 patches and update the patch to v31.

This new version patch includes the fix of libpq.smgl fix and the addition of regression test mode.
In libpq.smgl, the symbol indicating the message direction has been corrected from "<" ">" to "B" "F" in the documentation.

From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
Sent: Sunday, March 28, 2021 4:28 AM

...

Maybe the easiest way is to have a new flag PQTRACE_REGRESS_MODE.

To prepare for regression test, I read Message Formats documentation.
https://www.postgresql.org/docs/current/protocol-message-formats.html

Following protocol messages have values that depend on the code of master at that time;
- BackendKeyData(B) ... includes backend PID and backend private key
- ErrorResponse(B) ... includes error message line number
- FunctionCall(F) ... includes function OID
- NoticeResponse(B) ... includes notice message line number
- NotificationResponse (B) ... includes backend PID
- ParameterDescription ... includes parameter OID
- Parse(F) ... includes parameter data type OID
- RowDescription(B) ... includes OIDs

I checked status of conn->pqTraceFlags to decide whether output version-dependent messages or not in above protocol message output functions.
In ErrorResponse and NoticeResponse, I skip string type message logging only when field type code is "L".
In my understanding, other field code message type does not depend on version.
So I didn't skip other code type's string messages.
And I also changed description of pqTraceSetFlags()
by changing PQTRACE_SUPPRESS_TIMESTAMPS flag to the PQTRACE_REGRESS_MODE flag.

Output of regress mode is following;

B 124 ErrorResponse S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L R "parseRelOptionsInternal" \x00
B 14 ParameterDescription 2

Output of non-regress mode is following;

2021-03-30 12:55:31.327913 B 124 ErrorResponse S "ERROR" V "ERROR" C "22023" M "unrecognized parameter "some_nonexistent_parameter"" F "reloptions.c" L "1447" R "parseRelOptionsInternal" \x00
2021-03-30 12:56:12.691617 B 14 ParameterDescription 2 25 701

Regards,
Aya Iwata

Attachments:

v31-libpq-trace-log.patchapplication/octet-stream; name=v31-libpq-trace-log.patchDownload
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index be674fb..6fd9554 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -6459,12 +6459,27 @@ PGContextVisibility PQsetErrorContextVisibility(PGconn *conn, PGContextVisibilit
 
     <listitem>
      <para>
-      Enables  tracing of the client/server communication to a debugging file stream.
+      Enables tracing of the client/server communication to a debugging file
+      stream.
 <synopsis>
 void PQtrace(PGconn *conn, FILE *stream);
 </synopsis>
      </para>
 
+     <para>
+      Each line consists of: an optional timestamp, a direction indicator
+      (<literal>F</literal> for messages from client to server
+      or <literal>B</literal> for messages from server to client),
+      message length, message type, and message contents.
+      Non-message contents fields (timestamp, direction, length and message type)
+      are separated by a tab. Message contents are separated by a space.
+      Protocol strings are enclosed in double quotes, while strings used as data
+      values are enclosed in single quotes.  Non-printable chars are printed as
+      hexadecimal escapes.
+      Further message-type-specific detail can be found in
+      <xref linkend="protocol-message-formats"/>.
+     </para>
+
      <note>
       <para>
        On Windows, if the <application>libpq</application> library and an application are
@@ -6479,6 +6494,29 @@ void PQtrace(PGconn *conn, FILE *stream);
     </listitem>
    </varlistentry>
 
+   <varlistentry id="libpq-PQtraceSetFlags">
+    <term><function>PQtraceSetFlags</function><indexterm><primary>PQtraceSetFlags</primary></indexterm></term>
+
+    <listitem>
+     <para>
+      Controls the tracing behavior of client/server communication.
+<synopsis>
+void PQtraceSetFlags(PGconn *conn, int flags);
+</synopsis>
+     </para>
+
+     <para>
+      <literal>flags</literal> contains flag bits describing the operating mode
+      of tracing.
+      If <literal>flags</literal> contains <literal>PQTRACE_REGRESS_MODE</literal>,
+      then the timestamp and version-dependent messages are not included when
+      printing each message.
+      This function must be called after calling <function>PQtrace</function>.
+     </para>
+
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="libpq-PQuntrace">
     <term><function>PQuntrace</function><indexterm><primary>PQuntrace</primary></indexterm></term>
 
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2aca882..0424523 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -41,6 +41,7 @@ OBJS = \
 	fe-secure.o \
 	legacy-pqsignal.o \
 	libpq-events.o \
+	libpq-trace.o \
 	pqexpbuffer.o \
 	fe-auth.o
 
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5c48c14..a00701f 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -183,3 +183,4 @@ PQenterPipelineMode       180
 PQexitPipelineMode        181
 PQpipelineSync            182
 PQpipelineStatus          183
+PQtraceSetFlags           184
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 53b354a..a90bdb8 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -6859,27 +6859,6 @@ PQsetErrorContextVisibility(PGconn *conn, PGContextVisibility show_context)
 	return old;
 }
 
-void
-PQtrace(PGconn *conn, FILE *debug_port)
-{
-	if (conn == NULL)
-		return;
-	PQuntrace(conn);
-	conn->Pfdebug = debug_port;
-}
-
-void
-PQuntrace(PGconn *conn)
-{
-	if (conn == NULL)
-		return;
-	if (conn->Pfdebug)
-	{
-		fflush(conn->Pfdebug);
-		conn->Pfdebug = NULL;
-	}
-}
-
 PQnoticeReceiver
 PQsetNoticeReceiver(PGconn *conn, PQnoticeReceiver proc, void *arg)
 {
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index f143b8b..03592bd 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -970,10 +970,6 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
 	pgParameterStatus *pstatus;
 	pgParameterStatus *prev;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "pqSaveParameterStatus: '%s' = '%s'\n",
-				name, value);
-
 	/*
 	 * Forget any old information about the parameter
 	 */
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index ce2d24b..54d6a76 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -84,9 +84,6 @@ pqGetc(char *result, PGconn *conn)
 
 	*result = conn->inBuffer[conn->inCursor++];
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> %c\n", *result);
-
 	return 0;
 }
 
@@ -100,9 +97,6 @@ pqPutc(char c, PGconn *conn)
 	if (pqPutMsgBytes(&c, 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> %c\n", c);
-
 	return 0;
 }
 
@@ -138,10 +132,6 @@ pqGets_internal(PQExpBuffer buf, PGconn *conn, bool resetbuffer)
 
 	conn->inCursor = ++inCursor;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend> \"%s\"\n",
-				buf->data);
-
 	return 0;
 }
 
@@ -167,9 +157,6 @@ pqPuts(const char *s, PGconn *conn)
 	if (pqPutMsgBytes(s, strlen(s) + 1, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> \"%s\"\n", s);
-
 	return 0;
 }
 
@@ -188,13 +175,6 @@ pqGetnchar(char *s, size_t len, PGconn *conn)
 
 	conn->inCursor += len;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -212,13 +192,6 @@ pqSkipnchar(size_t len, PGconn *conn)
 	if (len > (size_t) (conn->inEnd - conn->inCursor))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "From backend (%lu)> ", (unsigned long) len);
-		fwrite(conn->inBuffer + conn->inCursor, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	conn->inCursor += len;
 
 	return 0;
@@ -234,13 +207,6 @@ pqPutnchar(const char *s, size_t len, PGconn *conn)
 	if (pqPutMsgBytes(s, len, conn))
 		return EOF;
 
-	if (conn->Pfdebug)
-	{
-		fprintf(conn->Pfdebug, "To backend> ");
-		fwrite(s, 1, len, conn->Pfdebug);
-		fprintf(conn->Pfdebug, "\n");
-	}
-
 	return 0;
 }
 
@@ -278,9 +244,6 @@ pqGetInt(int *result, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "From backend (#%lu)> %d\n", (unsigned long) bytes, *result);
-
 	return 0;
 }
 
@@ -314,9 +277,6 @@ pqPutInt(int value, size_t bytes, PGconn *conn)
 			return EOF;
 	}
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend (%lu#)> %d\n", (unsigned long) bytes, value);
-
 	return 0;
 }
 
@@ -525,10 +485,6 @@ pqPutMsgStart(char msg_type, PGconn *conn)
 	conn->outMsgEnd = endPos;
 	/* length word, if needed, will be filled in by pqPutMsgEnd */
 
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg %c\n",
-				msg_type ? msg_type : ' ');
-
 	return 0;
 }
 
@@ -563,10 +519,6 @@ pqPutMsgBytes(const void *buf, size_t len, PGconn *conn)
 int
 pqPutMsgEnd(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fprintf(conn->Pfdebug, "To backend> Msg complete, length %u\n",
-				conn->outMsgEnd - conn->outCount);
-
 	/* Fill in length word if needed */
 	if (conn->outMsgStart >= 0)
 	{
@@ -576,6 +528,16 @@ pqPutMsgEnd(PGconn *conn)
 		memcpy(conn->outBuffer + conn->outMsgStart, &msgLen, 4);
 	}
 
+	/* Trace message only when there is first 1 byte */
+	if (conn->Pfdebug)
+	{
+		if (conn->outCount < conn->outMsgStart)
+			pqTraceOutputMessage(conn, conn->outBuffer + conn->outCount, true);
+		else
+			pqTraceOutputNoTypeByteMessage(conn,
+										conn->outBuffer + conn->outMsgStart);
+	}
+
 	/* Make message eligible to send */
 	conn->outCount = conn->outMsgEnd;
 
@@ -1002,11 +964,13 @@ pqSendSome(PGconn *conn, int len)
 int
 pqFlush(PGconn *conn)
 {
-	if (conn->Pfdebug)
-		fflush(conn->Pfdebug);
-
 	if (conn->outCount > 0)
+	{
+		if (conn->Pfdebug)
+			fflush(conn->Pfdebug);
+
 		return pqSendSome(conn, conn->outCount);
+	}
 
 	return 0;
 }
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 306e89a..de77c06 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -457,6 +457,9 @@ pqParseInput3(PGconn *conn)
 		/* Successfully consumed this message */
 		if (conn->inCursor == conn->inStart + 5 + msgLength)
 		{
+			if(conn->Pfdebug)
+				pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 			/* Normal case: parsing agrees with specified length */
 			conn->inStart = conn->inCursor;
 		}
@@ -1660,6 +1663,9 @@ getCopyDataMessage(PGconn *conn)
 				return -1;
 		}
 
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
+
 		/* Drop the processed message and loop around for another */
 		conn->inStart = conn->inCursor;
 	}
@@ -2121,6 +2127,8 @@ pqFunctionCall3(PGconn *conn, Oid fnid,
 		}
 		/* Completed this message, keep going */
 		/* trust the specified message length as what to skip */
+		if(conn->Pfdebug)
+			pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
 		conn->inStart += 5 + msgLength;
 		needInput = false;
 	}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index cee42d4..3a2df4f 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -376,7 +376,9 @@ extern PGContextVisibility PQsetErrorContextVisibility(PGconn *conn,
 													   PGContextVisibility show_context);
 
 /* Enable/disable tracing */
+#define PQTRACE_REGRESS_MODE		1
 extern void PQtrace(PGconn *conn, FILE *debug_port);
+extern void PQtraceSetFlags(PGconn *conn, int flags);
 extern void PQuntrace(PGconn *conn);
 
 /* Override default notice handling routines */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6374ec6..1a1d0e1 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,7 @@ struct pg_conn
 
 	/* Optional file to write trace info to */
 	FILE	   *Pfdebug;
+	int			traceFlags;
 
 	/* Callback procedures for notice message processing */
 	PGNoticeHooks noticeHooks;
@@ -818,6 +819,12 @@ extern ssize_t pg_GSS_write(PGconn *conn, const void *ptr, size_t len);
 extern ssize_t pg_GSS_read(PGconn *conn, void *ptr, size_t len);
 #endif
 
+/* === in libpq-trace.c === */
+
+extern void pqTraceOutputMessage(PGconn *conn, const char *message,
+															bool toServer);
+extern void pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message);
+
 /* === miscellaneous macros === */
 
 /*
diff --git a/src/interfaces/libpq/libpq-trace.c b/src/interfaces/libpq/libpq-trace.c
new file mode 100644
index 0000000..fd5de2f
--- /dev/null
+++ b/src/interfaces/libpq/libpq-trace.c
@@ -0,0 +1,721 @@
+/*-------------------------------------------------------------------------
+ *
+ *	libpq-trace.c
+ *	  functions for libpq protocol tracing
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/interfaces/libpq/libpq-trace.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+#include <time.h>
+
+#ifdef WIN32
+#include "win32.h"
+#else
+#include <unistd.h>
+#include <sys/time.h>
+#endif
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "pgtime.h"
+#include "port/pg_bswap.h"
+
+void
+PQtrace(PGconn *conn, FILE *debug_port)
+{
+	if (conn == NULL)
+		return;
+	PQuntrace(conn);
+	if (debug_port == NULL)
+		return;
+
+	setvbuf(debug_port, NULL, _IOLBF, 0);
+	conn->Pfdebug = debug_port;
+	conn->traceFlags = 0;
+}
+
+void
+PQuntrace(PGconn *conn)
+{
+	if (conn == NULL)
+		return;
+	if (conn->Pfdebug)
+	{
+		fflush(conn->Pfdebug);
+		conn->Pfdebug = NULL;
+	}
+
+	conn->traceFlags = 0;
+}
+
+void
+PQtraceSetFlags(PGconn *conn, int flags)
+{
+	if (conn == NULL)
+		return;
+	/* If PQtrace() failed, do nothing. */
+	if (conn->Pfdebug == NULL)
+		return;
+	conn->traceFlags = flags;
+}
+
+/*
+ * Print the current time, with microseconds, into a caller-supplied
+ * buffer.
+ * Cribbed from setup_formatted_log_time, but much simpler.
+ */
+static void
+pqTraceFormatTimestamp(char *timestr, size_t ts_len)
+{
+	struct timeval tval;
+	pg_time_t	stamp_time;
+
+	gettimeofday(&tval, NULL);
+	stamp_time = (pg_time_t) tval.tv_sec;
+
+	strftime(timestr, ts_len,
+			 "%Y-%m-%d %H:%M:%S",
+			 localtime(&stamp_time));
+	/* append microseconds */
+	sprintf(timestr + strlen(timestr), ".%06d", (int) (tval.tv_usec));
+}
+
+/*
+ *   pqTraceOutputByte1: output 1 char message to the log
+ */
+static void
+pqTraceOutputByte1(FILE *pfdebug, const char *data, int *cursor)
+{
+	const char *v = data + *cursor;
+
+	/*
+	 * Show non-printable data in hex format, including the
+	 * terminating \0 that completes ErrorResponse and NoticeResponse
+	 * messages.
+	 */
+	if (!isprint(*v))
+		fprintf(pfdebug, " \\x%02x", *v);
+	else
+		fprintf(pfdebug, " %c", *v);
+	*cursor += 1;
+}
+
+/*
+ *   pqTraceOutputInt16: output a 2-byte integer message to the log
+ */
+static int
+pqTraceOutputInt16(FILE *pfdebug, const char *data, int *cursor)
+{
+	uint16		tmp;
+	int			result;
+
+	memcpy(&tmp, data + *cursor , 2);
+	*cursor += 2;
+	result = (int) pg_ntoh16(tmp);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputInt32: output a 4-byte integer message to the log
+ */
+static int
+pqTraceOutputInt32(FILE *pfdebug, const char *data, int *cursor)
+{
+	int			result;
+
+	memcpy(&result, data + *cursor, 4);
+	*cursor += 4;
+	result = (int) pg_ntoh32(result);
+	fprintf(pfdebug, " %d", result);
+
+	return result;
+}
+
+/*
+ *   pqTraceOutputString: output a string message to the log
+ */
+static void
+pqTraceOutputString(FILE *pfdebug, const char *data, int *cursor)
+{
+	int	len;
+
+	len = fprintf(pfdebug, " \"%s\"", data + *cursor);
+
+	/*
+	 * This is null-terminated string. So add 1 after subtracting 3
+	 * which is the double quotes and space length from len.
+	 */
+	*cursor += (len - 3 + 1);
+}
+
+/*
+ * pqTraceOutputNchar: output a string of exactly len bytes message to the log
+ */
+static void
+pqTraceOutputNchar(FILE *pfdebug, int len, const char *data, int *cursor)
+{
+	int			i,
+				next;			/* first char not yet printed */
+	const char	*v = data + *cursor;
+
+	fprintf(pfdebug, " \'");
+
+	for (next = i = 0; i < len; ++i)
+	{
+		if (isprint(v[i]))
+			continue;
+		else
+		{
+			fwrite(v + next, 1, i - next, pfdebug);
+			fprintf(pfdebug, "\\x%02x", v[i]);
+			next = i + 1;
+		}
+	}
+	if (next < len)
+		fwrite(v + next, 1, len - next, pfdebug);
+
+	fprintf(pfdebug, "\'");
+	*cursor += len;
+}
+
+/*
+ * Output functions by protocol message type
+ */
+
+/* Authentication */
+static void
+pqTraceOutputR(FILE *f, const char *message, int *cursor)
+{
+	fprintf(f, "Authentication\t");
+	pqTraceOutputInt32(f, message, cursor);
+}
+
+/* BackendKeyData */
+static void
+pqTraceOutputK(FILE *f, int traceFlags, const char *message, int *cursor)
+{
+	fprintf(f, "BackendKeyData\t");
+	if (traceFlags && PQTRACE_REGRESS_MODE)
+		*cursor += 4;
+	else
+		pqTraceOutputInt32(f, message, cursor);
+
+	if (traceFlags && PQTRACE_REGRESS_MODE)
+		*cursor += 4;
+	else
+		pqTraceOutputInt32(f, message, cursor);
+}
+
+/* Bind */
+static void
+pqTraceOutputB(FILE *f, const char *message, int *cursor)
+{
+	int nparams;
+
+	fprintf(f, "Bind\t");
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt16(f, message, cursor);
+
+	nparams = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nparams; i++)
+	{
+		int nbytes;
+
+		nbytes = pqTraceOutputInt32(f, message, cursor);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(f, nbytes, message, cursor);
+	}
+
+	nparams = pqTraceOutputInt16(f, message, cursor);
+	for (int i = 0; i < nparams; i++)
+		pqTraceOutputInt16(f, message, cursor);
+}
+
+/* Close(F) or CommandComplete(B) */
+static void
+pqTraceOutputC(FILE *f, bool toServer, const char *message, int *cursor)
+{
+	if (toServer)
+	{
+		fprintf(f, "Close\t");
+		pqTraceOutputByte1(f, message, cursor);
+		pqTraceOutputString(f, message, cursor);
+	}
+	else
+	{
+		fprintf(f, "CommandComplete\t");
+		pqTraceOutputString(f, message, cursor);
+	}
+}
+
+/* CopyFail */
+static void
+pqTraceOutputf(FILE *f, const char *message, int *cursor)
+{
+	fprintf(f, "CopyFail\t");
+	pqTraceOutputString(f, message, cursor);
+}
+
+/* CopyInResponse */
+static void
+pqTraceOutputG(FILE *f, const char *message, int *cursor)
+{
+	int	nfields;
+
+	fprintf(f, "CopyInResponse\t");
+	pqTraceOutputByte1(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
+}
+
+/* CopyOutResponse */
+static void
+pqTraceOutputH(FILE *f, const char *message, int *cursor)
+{
+	int	nfields;
+
+	fprintf(f, "CopyOutResponse\t");
+	pqTraceOutputByte1(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
+}
+
+/* CopyBothResponse */
+static void
+pqTraceOutputW(FILE *f, const char *message, int *cursor, int length)
+{
+	fprintf(f, "CopyBothResponse\t");
+	pqTraceOutputByte1(f, message, cursor);
+
+	while (length > *cursor)
+		pqTraceOutputInt16(f, message, cursor);
+}
+
+/* Describe(F) or DataRow(B) */
+static void
+pqTraceOutputD(FILE *f, bool toServer, const char *message, int *cursor)
+{
+	if (toServer)
+	{
+		fprintf(f, "Describe\t");
+		pqTraceOutputByte1(f, message, cursor);
+		pqTraceOutputString(f, message, cursor);
+	}
+	else
+	{
+		int		nfields;
+		int		len;
+		int		i;
+
+		fprintf(f, "DataRow\t");
+		nfields = pqTraceOutputInt16(f, message, cursor);
+		for (i = 0; i < nfields; i++)
+		{
+			len = pqTraceOutputInt32(f, message, cursor);
+			if (len == -1)
+				continue;
+			pqTraceOutputNchar(f, len, message, cursor);
+		}
+	}
+}
+
+/* Execute(F) or ErrorResponse(B) */
+static void
+pqTraceOutputE(FILE *f, int traceFlags, bool toServer, const char *message, int *cursor, int length)
+{
+	if (toServer)
+	{
+		fprintf(f, "Execute\t");
+		pqTraceOutputString(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+	}
+	else
+	{
+		fprintf(f, "ErrorResponse\t");
+		for (;;)
+		{
+			pqTraceOutputByte1(f, message, cursor);
+			if (message[*cursor - 1] == '\0')
+				break;
+			if (message[*cursor - 1] == 'L' && traceFlags && PQTRACE_REGRESS_MODE)
+				*cursor += strlen(message + *cursor) + 1;
+			else
+				pqTraceOutputString(f, message, cursor);
+		}
+	}
+}
+
+/* FunctionCall */
+static void
+pqTraceOutputF(FILE *f, int traceFlags, const char *message, int *cursor)
+{
+	int nfields;
+	int nbytes;
+
+	fprintf(f, "FunctionCall\t");
+	if (traceFlags && PQTRACE_REGRESS_MODE)
+		*cursor += 4;
+	else
+		pqTraceOutputInt32(f, message, cursor);
+	nfields = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nfields; i++)
+		pqTraceOutputInt16(f, message, cursor);
+
+	nfields = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nfields; i++)
+	{
+		nbytes = pqTraceOutputInt32(f, message, cursor);
+		if (nbytes == -1)
+			continue;
+		pqTraceOutputNchar(f, nbytes, message, cursor);
+	}
+
+	pqTraceOutputInt16(f, message, cursor);
+}
+
+/* FunctionCallResponse */
+static void
+pqTraceOutputV(FILE *f, const char *message, int *cursor)
+{
+	int		len;
+
+	fprintf(f, "FunctionCallResponse\t");
+	len = pqTraceOutputInt32(f, message, cursor);
+	if (len != -1)
+		pqTraceOutputNchar(f, len, message, cursor);
+}
+
+/* NegotiateProtocolVersion */
+static void
+pqTraceOutputv(FILE *f, const char *message, int *cursor)
+{
+	fprintf(f, "NegotiateProtocolVersion\t");
+	pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputInt32(f, message, cursor);
+}
+
+/* NoticeResponse */
+static void
+pqTraceOutputN(FILE *f, int traceFlags, const char *message, int *cursor, int length)
+{
+	fprintf(f, "NoticeResponse\t");
+	for (;;)
+	{
+		pqTraceOutputByte1(f, message, cursor);
+		if (message[*cursor - 1] == '\0')
+			break;
+		if (message[*cursor - 1] == 'L' && traceFlags && PQTRACE_REGRESS_MODE)
+			*cursor += strlen(message + *cursor) + 1;
+		else
+			pqTraceOutputString(f, message, cursor);
+	}
+}
+
+/* NotificationResponse */
+static void
+pqTraceOutputA(FILE *f, int traceFlags, const char *message, int *cursor)
+{
+	fprintf(f, "NotificationResponse\t");
+	if (traceFlags && PQTRACE_REGRESS_MODE)
+		*cursor += 4;
+	else
+		pqTraceOutputInt32(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+}
+
+/* ParameterDescription */
+static void
+pqTraceOutputt(FILE *f, int traceFlags, const char *message, int *cursor)
+{
+	int	nfields;
+
+	fprintf(f, "ParameterDescription\t");
+	nfields = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nfields; i++)
+	{
+		if (traceFlags && PQTRACE_REGRESS_MODE)
+			*cursor += 4;
+		else
+			pqTraceOutputInt32(f, message, cursor);
+	}
+}
+
+/* ParameterStatus */
+static void
+pqTraceOutputS(FILE *f, const char *message, int *cursor)
+{
+	fprintf(f, "ParameterStatus\t");
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+}
+
+/* Parse */
+static void
+pqTraceOutputP(FILE *f, int traceFlags, const char *message, int *cursor)
+{
+	int nparams;
+
+	fprintf(f, "Parse\t");
+	pqTraceOutputString(f, message, cursor);
+	pqTraceOutputString(f, message, cursor);
+	nparams = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nparams; i++)
+	{
+		if (traceFlags && PQTRACE_REGRESS_MODE)
+			*cursor += 4;
+		else
+			pqTraceOutputInt32(f, message, cursor);
+	}
+}
+
+/* Query */
+static void
+pqTraceOutputQ(FILE *f, const char *message, int *cursor)
+{
+	fprintf(f, "Query\t");
+	pqTraceOutputString(f, message, cursor);
+}
+
+/* ReadyForQuery */
+static void
+pqTraceOutputZ(FILE *f, const char *message, int *cursor)
+{
+	fprintf(f, "ReadyForQuery\t");
+	pqTraceOutputByte1(f, message, cursor);
+}
+
+/* RowDescription */
+static void
+pqTraceOutputT(FILE *f, int traceFlags, const char *message, int *cursor)
+{
+	int nfields;
+
+	fprintf(f, "RowDescription\t");
+	nfields = pqTraceOutputInt16(f, message, cursor);
+
+	for (int i = 0; i < nfields; i++)
+	{
+		pqTraceOutputString(f, message, cursor);
+		if (traceFlags && PQTRACE_REGRESS_MODE)
+			*cursor += 4;
+		else
+			pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+		if (traceFlags && PQTRACE_REGRESS_MODE)
+			*cursor += 4;
+		else
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+		pqTraceOutputInt32(f, message, cursor);
+		pqTraceOutputInt16(f, message, cursor);
+	}
+}
+
+void
+pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
+{
+	char		timestr[128];
+	char 		id;
+	int			length;
+	char	   *prefix = toServer ? "F" : "B";
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_REGRESS_MODE) == 0)
+	{
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+		fprintf(conn->Pfdebug, "%s\t", timestr);
+	}
+
+	id = message[LogCursor++];
+
+	memcpy(&length, message + LogCursor, 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
+
+	switch(id)
+	{
+		case 'R':	/* Authentication */
+			pqTraceOutputR(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'K':	/* secret key data from the backend */
+			pqTraceOutputK(conn->Pfdebug, conn->traceFlags, message, &LogCursor);
+			break;
+		case 'B':	/* Bind */
+			pqTraceOutputB(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'C':	/* Close(F) or Command Complete(B) */
+			pqTraceOutputC(conn->Pfdebug, toServer, message, &LogCursor);
+			break;
+		case 'd':	/* Copy Data */
+			/* Drop COPY data to reduce the overhead of logging. */
+			break;
+		case 'f':	/* Copy Fail */
+			pqTraceOutputf(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'G':	/* Start Copy In */
+			pqTraceOutputG(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'H':	/* Flush(F) or Start Copy Out(B) */
+			if (!toServer)
+				pqTraceOutputH(conn->Pfdebug, message, &LogCursor);
+			else
+				fprintf(conn->Pfdebug, "Flush");
+			break;
+		case 'W':	/* Start Copy Both */
+			pqTraceOutputW(conn->Pfdebug, message, &LogCursor, length);
+			break;
+		case 'D':	/* Describe(F) or Data Row(B) */
+			pqTraceOutputD(conn->Pfdebug, toServer, message, &LogCursor);
+			break;
+		case 'E':	/* Execute(F) or Error Response(B) */
+			pqTraceOutputE(conn->Pfdebug, conn->traceFlags, toServer, message, &LogCursor, length);
+			break;
+		case 'F':	/* Function Call */
+			pqTraceOutputF(conn->Pfdebug, conn->traceFlags, message, &LogCursor);
+			break;
+		case 'V':	/* Function Call response */
+			pqTraceOutputV(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'v':	/* Negotiate Protocol Version */
+			pqTraceOutputv(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'N':	/* Notice Response */
+			pqTraceOutputN(conn->Pfdebug, conn->traceFlags, message, &LogCursor, length);
+			break;
+		case 'A':	/* Notification Response */
+			pqTraceOutputA(conn->Pfdebug, conn->traceFlags, message, &LogCursor);
+			break;
+		case 't':	/* Parameter Description */
+			pqTraceOutputt(conn->Pfdebug, conn->traceFlags, message, &LogCursor);
+			break;
+		case 'S':	/* Parameter Status(B) or Sync(F) */
+			if (!toServer)
+				pqTraceOutputS(conn->Pfdebug, message, &LogCursor);
+			else
+				fprintf(conn->Pfdebug, "Sync");
+			break;
+		case 'P':	/* Parse */
+			pqTraceOutputP(conn->Pfdebug, conn->traceFlags, message, &LogCursor);
+			break;
+		case 'Q':	/* Query */
+			pqTraceOutputQ(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'Z':	/* Ready For Query */
+			pqTraceOutputZ(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 'T':	/* Row Description */
+			pqTraceOutputT(conn->Pfdebug, conn->traceFlags, message, &LogCursor);
+			break;
+		case '2':	/* Bind Complete */
+			fprintf(conn->Pfdebug, "BindComplete");
+			/* No message content */
+			break;
+		case '3':	/* Close Complete */
+			fprintf(conn->Pfdebug, "CloseComplete");
+			/* No message content */
+			break;
+		case 'c':	/* Copy Done */
+			fprintf(conn->Pfdebug, "CopyDone");
+			/* No message content */
+			break;
+		case 'I':	/* Empty Query Response */
+			fprintf(conn->Pfdebug, "EmptyQueryResponse");
+			/* No message content */
+			break;
+		case 'n':	/* No Data */
+			fprintf(conn->Pfdebug, "NoData");
+			/* No message content */
+			break;
+		case '1':	/* Parse Complete */
+			fprintf(conn->Pfdebug, "ParseComplete");
+			/* No message content */
+			break;
+		case 's':	/* Portal Suspended */
+			fprintf(conn->Pfdebug, "PortalSuspended");
+			/* No message content */
+			break;
+		case 'X':	/* Terminate */
+			fprintf(conn->Pfdebug, "Terminate");
+			/* No message content */
+			break;
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: %02x", id);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+
+	/*
+	 * Verify the printing routine did it right.  Note that the one-byte
+	 * message identifier is not included in the length, but our cursor
+	 * does include it.
+	 */
+	if (LogCursor - 1 != length)
+		fprintf(conn->Pfdebug,
+				"mismatched message length: consumed %d, expected %d\n",
+				LogCursor - 1, length);
+}
+
+void
+pqTraceOutputNoTypeByteMessage(PGconn *conn, const char *message)
+{
+	char		timestr[128];
+	int			length;
+	int			LogCursor = 0;
+
+	if ((conn->traceFlags & PQTRACE_REGRESS_MODE) == 0)
+		pqTraceFormatTimestamp(timestr, sizeof(timestr));
+	else
+		timestr[0] = '\0';
+
+	memcpy(&length, message + LogCursor , 4);
+	length = (int) pg_ntoh32(length);
+	LogCursor += 4;
+
+	fprintf(conn->Pfdebug, "%s\t>\t%d\t", timestr, length);
+
+	switch(length)
+	{
+		case 16:	/* CancelRequest */
+			fprintf(conn->Pfdebug, "CancelRequest\t");
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			pqTraceOutputInt32(conn->Pfdebug, message, &LogCursor);
+			break;
+		case 8 :	/* GSSENCRequest or SSLRequest */
+			/* These messages do not reach here. */
+		default:
+			fprintf(conn->Pfdebug, "Unknown message: length is %d", length);
+			break;
+	}
+
+	fputc('\n', conn->Pfdebug);
+}
#192'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: iwata.aya@fujitsu.com (#191)
Re: libpq debug log

Okay, pushed this patch and the new testing for it based on
libpq_pipeline. We'll see how the buildfarm likes it.

I made some further changes to the last version; user-visibly, I split
the trace flags in two, keeping the timestamp suppression separate from
the redacting feature for regression testing.

I didn't like the idea of silently skipping the redacted fields, so I
changed the code to print NNNN or SSSS instead. I also made the
redacting occur in the last mile (pqTraceOutputInt32 / String) rather
that in their callers: it seemed quite odd to advance the cursor in the
"else" branch.

I refactored the duplicate code that appeared for Notice and Error.
In that function, we redact not only the 'L' field (what Iwata-san was
doing) but also 'F' (file) and 'R' (routine) because those things can
move around for reasons that are not interesting to testing this code.

In the libpq_pipeline commit I added 'pipeline_abort' and 'transaction'
to the cases that generate traces, which adds coverage for
NoticeResponse and ErrorResponse.

--
�lvaro Herrera 39�49'30"S 73�17'W

#193'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: 'alvherre@alvh.no-ip.org' (#192)
Re: libpq debug log

So crake failed. The failure is that it doesn't print the DataRow
messages. That's quite odd. We see this in the trace log:

Mar 30 20:05:15 # F 54 Parse "" "SELECT 1.0/g FROM generate_series(3, -1, -1) g" 0
Mar 30 20:05:15 # F 12 Bind "" "" 0 0 0
Mar 30 20:05:15 # F 6 Describe P ""
Mar 30 20:05:15 # F 9 Execute "" 0
Mar 30 20:05:15 # F 4 Sync
Mar 30 20:05:15 # B 4 ParseComplete
Mar 30 20:05:15 # B 4 BindComplete
Mar 30 20:05:15 # B 33 RowDescription 1 "?column?" NNNN 0 NNNN 65535 -1 0
Mar 30 20:05:15 # B 70 ErrorResponse S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \\x00
Mar 30 20:05:15 # B 5 ReadyForQuery I

and the expected is this:

Mar 30 20:05:15 # F 54 Parse "" "SELECT 1.0/g FROM generate_series(3, -1, -1) g" 0
Mar 30 20:05:15 # F 12 Bind "" "" 0 0 0
Mar 30 20:05:15 # F 6 Describe P ""
Mar 30 20:05:15 # F 9 Execute "" 0
Mar 30 20:05:15 # F 4 Sync
Mar 30 20:05:15 # B 4 ParseComplete
Mar 30 20:05:15 # B 4 BindComplete
Mar 30 20:05:15 # B 33 RowDescription 1 "?column?" NNNN 0 NNNN 65535 -1 0
Mar 30 20:05:15 # B 32 DataRow 1 22 '0.33333333333333333333'
Mar 30 20:05:15 # B 32 DataRow 1 22 '0.50000000000000000000'
Mar 30 20:05:15 # B 32 DataRow 1 22 '1.00000000000000000000'
Mar 30 20:05:15 # B 70 ErrorResponse S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \\x00
Mar 30 20:05:15 # B 5 ReadyForQuery I

...

I wonder if this is a libpq pipeline problem rather than a PQtrace
problem. In that test case we're using the libpq singlerow mode, so we
should see three rows before the error is sent, but for some reason
crake is not doing that.

aborted pipeline... NOTICE: table "pq_pipeline_demo" does not exist, skipping
ok
got expected ERROR: cannot insert multiple commands into a prepared statement
got expected division-by-zero
ok 5 - libpq_pipeline pipeline_abort
not ok 6 - pipeline_abort trace match

Other hosts seem to get it right:

# Running: libpq_pipeline -t /Users/buildfarm/bf-data/HEAD/pgsql.build/src/test/modules/libpq_pipeline/tmp_check/log/pipeline_abort.trace pipeline_abort port=49797 host=/var/folders/md/8mp8j5m5169ccgy11plb4w_80000gp/T/iA9YP9_IHa dbname='postgres' 10000
aborted pipeline... NOTICE: table "pq_pipeline_demo" does not exist, skipping
ok
got expected ERROR: cannot insert multiple commands into a prepared statement
got row: 0.33333333333333333333
got row: 0.50000000000000000000
got row: 1.00000000000000000000
got expected division-by-zero
ok 5 - libpq_pipeline pipeline_abort
ok 6 - pipeline_abort trace match

--
�lvaro Herrera Valdivia, Chile
"This is what I like so much about PostgreSQL. Most of the surprises
are of the "oh wow! That's cool" Not the "oh shit!" kind. :)"
Scott Marlowe, http://archives.postgresql.org/pgsql-admin/2008-10/msg00152.php

#194tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: 'alvherre@alvh.no-ip.org' (#192)
RE: libpq debug log

From: 'alvherre@alvh.no-ip.org' <alvherre@alvh.no-ip.org>

Okay, pushed this patch and the new testing for it based on
libpq_pipeline. We'll see how the buildfarm likes it.

Thank you very much! I appreciate you taking your valuable time while I imagine you must be pretty busy with taking care of other (possibly more important) patches.

TBH, when Tom-san suggested drastic change, I was afraid we may not be able to complete this in PG 14. But in the end, I'm very happy that the patch has become much leaner and cleaner.

And congratulations on your first commit, Iwata-san! I hope you can have time and energy to try adding a connection parameter to enable tracing, which eliminates application modification.

I didn't like the idea of silently skipping the redacted fields, so I
changed the code to print NNNN or SSSS instead. I also made the
redacting occur in the last mile (pqTraceOutputInt32 / String) rather
that in their callers: it seemed quite odd to advance the cursor in the
"else" branch.

I refactored the duplicate code that appeared for Notice and Error.
In that function, we redact not only the 'L' field (what Iwata-san was
doing) but also 'F' (file) and 'R' (routine) because those things can
move around for reasons that are not interesting to testing this code.

In the libpq_pipeline commit I added 'pipeline_abort' and 'transaction'
to the cases that generate traces, which adds coverage for
NoticeResponse and ErrorResponse.

These make sense to me. Thank you for repeatedly polishing and making the patch better much.

Regards
Takayuki Tsunakawa

#195'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: 'alvherre@alvh.no-ip.org' (#193)
Re: libpq debug log

On 2021-Mar-30, 'alvherre@alvh.no-ip.org' wrote:

got expected ERROR: cannot insert multiple commands into a prepared statement
got expected division-by-zero

Eh? this is not at all expected, of course, but clearly not PQtrace's
fault. I'll look tomorrow.

--
�lvaro Herrera 39�49'30"S 73�17'W
"El sentido de las cosas no viene de las cosas, sino de
las inteligencias que las aplican a sus problemas diarios
en busca del progreso." (Ernesto Hern�ndez-Novich)

#196tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com
In reply to: 'alvherre@alvh.no-ip.org' (#195)
RE: libpq debug log

From: 'alvherre@alvh.no-ip.org' <alvherre@alvh.no-ip.org>

got expected ERROR: cannot insert multiple commands into a prepared

statement

got expected division-by-zero

Eh? this is not at all expected, of course, but clearly not PQtrace's
fault. I'll look tomorrow.

Iwata-san and I were starting to investigate the issue. I guessed the first message was not expected. Please let us know if there's something we can help.

(I was amazed that you finally committed this great feature, libpq pipeline, while you are caring about many patches.)

Regards
Takayuki Tsunakawa

#197Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#193)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

So crake failed. The failure is that it doesn't print the DataRow
messages. That's quite odd. We see this in the trace log:

I think this is a timing problem that's triggered (on some machines)
by force_parallel_mode = regress. Looking at spurfowl's latest
failure of this type, the postmaster log shows

2021-03-31 14:34:54.982 EDT [18233:15] 001_libpq_pipeline.pl LOG: execute <unnamed>: SELECT 1.0/g FROM generate_series(3, -1, -1) g
2021-03-31 14:34:54.992 EDT [18234:1] ERROR: division by zero
2021-03-31 14:34:54.992 EDT [18234:2] STATEMENT: SELECT 1.0/g FROM generate_series(3, -1, -1) g
2021-03-31 14:34:54.993 EDT [18233:16] 001_libpq_pipeline.pl ERROR: division by zero
2021-03-31 14:34:54.993 EDT [18233:17] 001_libpq_pipeline.pl STATEMENT: SELECT 1.0/g FROM generate_series(3, -1, -1) g
2021-03-31 14:34:54.995 EDT [18216:4] LOG: background worker "parallel worker" (PID 18234) exited with exit code 1
2021-03-31 14:34:54.995 EDT [18233:18] 001_libpq_pipeline.pl LOG: could not send data to client: Broken pipe
2021-03-31 14:34:54.995 EDT [18233:19] 001_libpq_pipeline.pl FATAL: connection to client lost

We can see that the division by zero occurred in a parallel worker.
My theory is that in parallel mode, it's uncertain whether the
error will be reported before or after the "preceding" successful
output rows. So you need to disable parallelism to make this
test case stable.

regards, tom lane

#198'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#197)
Re: libpq debug log

On 2021-Mar-31, Tom Lane wrote:

I think this is a timing problem that's triggered (on some machines)
by force_parallel_mode = regress. Looking at spurfowl's latest
failure of this type, the postmaster log shows

2021-03-31 14:34:54.982 EDT [18233:15] 001_libpq_pipeline.pl LOG: execute <unnamed>: SELECT 1.0/g FROM generate_series(3, -1, -1) g
2021-03-31 14:34:54.992 EDT [18234:1] ERROR: division by zero
2021-03-31 14:34:54.992 EDT [18234:2] STATEMENT: SELECT 1.0/g FROM generate_series(3, -1, -1) g
2021-03-31 14:34:54.993 EDT [18233:16] 001_libpq_pipeline.pl ERROR: division by zero
2021-03-31 14:34:54.993 EDT [18233:17] 001_libpq_pipeline.pl STATEMENT: SELECT 1.0/g FROM generate_series(3, -1, -1) g
2021-03-31 14:34:54.995 EDT [18216:4] LOG: background worker "parallel worker" (PID 18234) exited with exit code 1
2021-03-31 14:34:54.995 EDT [18233:18] 001_libpq_pipeline.pl LOG: could not send data to client: Broken pipe
2021-03-31 14:34:54.995 EDT [18233:19] 001_libpq_pipeline.pl FATAL: connection to client lost

We can see that the division by zero occurred in a parallel worker.
My theory is that in parallel mode, it's uncertain whether the
error will be reported before or after the "preceding" successful
output rows. So you need to disable parallelism to make this
test case stable.

Ooh, that makes sense, thanks. Added a SET in the program itself to set
it to off.

This is not the *only* issue though; at least animal drongo shows a
completely different failure, where the last few tests don't even get to
run, and the server log just has this:

2021-03-31 20:20:14.460 UTC [178364:4] 001_libpq_pipeline.pl LOG: disconnection: session time: 0:00:00.469 user=pgrunner database=postgres host=127.0.0.1 port=52638
2021-03-31 20:20:14.681 UTC [178696:1] [unknown] LOG: connection received: host=127.0.0.1 port=52639
2021-03-31 20:20:14.687 UTC [178696:2] [unknown] LOG: connection authorized: user=pgrunner database=postgres application_name=001_libpq_pipeline.pl
2021-03-31 20:20:15.157 UTC [178696:3] 001_libpq_pipeline.pl LOG: could not receive data from client: An existing connection was forcibly closed by the remote host.

I suppose the program is crashing for some reason.

--
�lvaro Herrera Valdivia, Chile
"Los dioses no protegen a los insensatos. �stos reciben protecci�n de
otros insensatos mejor dotados" (Luis Wu, Mundo Anillo)

#199Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#198)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

This is not the *only* issue though; at least animal drongo shows a
completely different failure, where the last few tests don't even get to
run, and the server log just has this:

That is weird - only test 4 (of 8) runs at all, the rest seem to
fail to connect. What's different about pipelined_insert?

regards, tom lane

#200Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#199)
Re: libpq debug log

I wrote:

That is weird - only test 4 (of 8) runs at all, the rest seem to
fail to connect. What's different about pipelined_insert?

Oh ... there's a pretty obvious theory. pipelined_insert is
the only one that is not asked to write a trace file.
So for some reason, opening the trace file fails.
(I wonder why we don't see an error message for this though.)

regards, tom lane

#201'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#200)
1 attachment(s)
Re: libpq debug log

On 2021-Mar-31, Tom Lane wrote:

I wrote:

That is weird - only test 4 (of 8) runs at all, the rest seem to
fail to connect. What's different about pipelined_insert?

Oh ... there's a pretty obvious theory. pipelined_insert is
the only one that is not asked to write a trace file.
So for some reason, opening the trace file fails.
(I wonder why we don't see an error message for this though.)

.. oh, I think we forgot to set conn->Pfdebug = NULL when creating the
connection. So when we do PQtrace(), the first thing it does is
PQuntrace(), and then that tries to do fflush(conn->Pfdebug) ---> crash.
So this should fix it.

--
�lvaro Herrera Valdivia, Chile
<inflex> really, I see PHP as like a strange amalgamation of C, Perl, Shell
<crab> inflex: you know that "amalgam" means "mixture with mercury",
more or less, right?
<crab> i.e., "deadly poison"

Attachments:

initialize.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index a90bdb8ab6..56a8266bc3 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -3952,6 +3952,7 @@ makeEmptyPGconn(void)
 	conn->verbosity = PQERRORS_DEFAULT;
 	conn->show_context = PQSHOW_CONTEXT_ERRORS;
 	conn->sock = PGINVALID_SOCKET;
+	conn->Pfdebug = NULL;
 
 	/*
 	 * We try to send at least 8K at a time, which is the usual size of pipe
#202'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: 'alvherre@alvh.no-ip.org' (#201)
Re: libpq debug log

On 2021-Mar-31, 'alvherre@alvh.no-ip.org' wrote:

.. oh, I think we forgot to set conn->Pfdebug = NULL when creating the
connection. So when we do PQtrace(), the first thing it does is
PQuntrace(), and then that tries to do fflush(conn->Pfdebug) ---> crash.
So this should fix it.

I tried to use MALLOC_PERTURB_ to prove this theory, but I failed at it.
I'll just push this blind and see what happens.

--
�lvaro Herrera 39�49'30"S 73�17'W
"Someone said that it is at least an order of magnitude more work to do
production software than a prototype. I think he is wrong by at least
an order of magnitude." (Brian Kernighan)

#203Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#201)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

On 2021-Mar-31, Tom Lane wrote:

So for some reason, opening the trace file fails.
(I wonder why we don't see an error message for this though.)

.. oh, I think we forgot to set conn->Pfdebug = NULL when creating the
connection. So when we do PQtrace(), the first thing it does is
PQuntrace(), and then that tries to do fflush(conn->Pfdebug) ---> crash.
So this should fix it.

Nope, see the MemSet a few lines further up. This change seems like
good style, but it won't fix anything --- else we'd have been
having issues with the pre-existing trace logic.

What I suspect is some Windows dependency in the way that
001_libpq_pipeline.pl is setting up the trace output files.
cc'ing Andrew to see if he has any ideas.

regards, tom lane

#204Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#203)
Re: libpq debug log

I wrote:

What I suspect is some Windows dependency in the way that
001_libpq_pipeline.pl is setting up the trace output files.

While this may have little to do with drongo's issue,
I'm going to take exception to this bit that I see that
the patch added to PQtrace():

/* Make the trace stream line-buffered */
setvbuf(debug_port, NULL, _IOLBF, 0);

I do not like that on two grounds:

1. The trace file handle belongs to the calling application,
not to libpq. I do not think libpq has any business editorializing
on the file's settings.

2. POSIX says that setvbuf() must be done before any other
operation on the file. Since we have no way to know whether
any output has already been written to the trace file, the
above is a spec violation.

... and as for an actual explanation for drongo's issue,
I'm sort of wondering why libpq_pipeline.c is opening the
trace file in "w+" mode rather than vanilla "w" mode.
That seems unnecessary, and maybe it doesn't work so well
on Windows.

regards, tom lane

#205'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#204)
Re: libpq debug log

On 2021-Mar-31, Tom Lane wrote:

While this may have little to do with drongo's issue,
I'm going to take exception to this bit that I see that
the patch added to PQtrace():

/* Make the trace stream line-buffered */
setvbuf(debug_port, NULL, _IOLBF, 0);

Mea culpa. I added this early on because it made PQtrace's use more
comfortable, but I agree that it's a mistake. Removed. I put it in
libpq_pipeline.c instead.

... and as for an actual explanation for drongo's issue,
I'm sort of wondering why libpq_pipeline.c is opening the
trace file in "w+" mode rather than vanilla "w" mode.
That seems unnecessary, and maybe it doesn't work so well
on Windows.

well, at least the MSVC documentation claims that it works, but who
knows. I removed that too.
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen?view=msvc-160

--
�lvaro Herrera Valdivia, Chile

#206'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#200)
Re: libpq debug log

On 2021-Mar-31, Tom Lane wrote:

I wrote:

That is weird - only test 4 (of 8) runs at all, the rest seem to
fail to connect. What's different about pipelined_insert?

Oh ... there's a pretty obvious theory. pipelined_insert is
the only one that is not asked to write a trace file.
So for some reason, opening the trace file fails.
(I wonder why we don't see an error message for this though.)

Eh, so I forgot to strdup(optarg[optind]). Apparently that works fine
in glibc but other getopt implementations are likely not so friendly.

--
�lvaro Herrera 39�49'30"S 73�17'W
"I am amazed at [the pgsql-sql] mailing list for the wonderful support, and
lack of hesitasion in answering a lost soul's question, I just wished the rest
of the mailing list could be like this." (Fotis)
(http://archives.postgresql.org/pgsql-sql/2006-06/msg00265.php)

#207Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#206)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

Eh, so I forgot to strdup(optarg[optind]). Apparently that works fine
in glibc but other getopt implementations are likely not so friendly.

Hmm ... interesting theory, but I don't think I buy it, since the
program isn't doing anything that should damage argv[]. I guess
we'll soon find out.

regards, tom lane

#208Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#206)
Re: libpq debug log

BTW, what in the world is this supposed to accomplish?

-                        (long long) rows_to_send);
+                        (1L << 62) + (long long) rows_to_send);

Various buildfarm members are complaining that the shift distance
is more than the width of "long", which I'd just fix with s/L/LL/
except that the addition of the constant seems just wrong to
begin with.

regards, tom lane

#209'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#208)
Re: libpq debug log

On 2021-Apr-01, Tom Lane wrote:

BTW, what in the world is this supposed to accomplish?

-                        (long long) rows_to_send);
+                        (1L << 62) + (long long) rows_to_send);

Various buildfarm members are complaining that the shift distance
is more than the width of "long", which I'd just fix with s/L/LL/
except that the addition of the constant seems just wrong to
begin with.

It makes the text representation wider, which means some buffer fills up
faster and the program switches from sending to receiving.

--
�lvaro Herrera Valdivia, Chile
"Ni a�n el genio muy grande llegar�a muy lejos
si tuviera que sacarlo todo de su propio interior" (Goethe)

#210Tom Lane
tgl@sss.pgh.pa.us
In reply to: 'alvherre@alvh.no-ip.org' (#209)
Re: libpq debug log

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

On 2021-Apr-01, Tom Lane wrote:

BTW, what in the world is this supposed to accomplish?
-                        (long long) rows_to_send);
+                        (1L << 62) + (long long) rows_to_send);

It makes the text representation wider, which means some buffer fills up
faster and the program switches from sending to receiving.

Hm. If the actual field value isn't relevant, why bother including
rows_to_send in it? A constant string would be simpler and much
less confusing as to its purpose.

regards, tom lane

#211Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#210)
Re: libpq debug log

So drongo is still failing, and after a bit of looking around at
other code I finally got hit with the clue hammer. Per port.h:

* On Windows, setvbuf() does not support _IOLBF mode, and interprets that
* as _IOFBF. To add insult to injury, setvbuf(file, NULL, _IOFBF, 0)
* crashes outright if "parameter validation" is enabled. Therefore, in
* places where we'd like to select line-buffered mode, we fall back to
* unbuffered mode instead on Windows. Always use PG_IOLBF not _IOLBF
* directly in order to implement this behavior.

You want to do the honors? And do something about that shift bug
while at it.

regards, tom lane

#212'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#210)
Re: libpq debug log

On 2021-Apr-01, Tom Lane wrote:

"'alvherre@alvh.no-ip.org'" <alvherre@alvh.no-ip.org> writes:

On 2021-Apr-01, Tom Lane wrote:

BTW, what in the world is this supposed to accomplish?
-                        (long long) rows_to_send);
+                        (1L << 62) + (long long) rows_to_send);

It makes the text representation wider, which means some buffer fills up
faster and the program switches from sending to receiving.

Hm. If the actual field value isn't relevant, why bother including
rows_to_send in it? A constant string would be simpler and much
less confusing as to its purpose.

Hah, yeah, we could as well do that I guess :-)

--
�lvaro Herrera Valdivia, Chile
"I think my standards have lowered enough that now I think 'good design'
is when the page doesn't irritate the living f*ck out of me." (JWZ)

#213'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#211)
Re: libpq debug log

On 2021-Apr-01, Tom Lane wrote:

So drongo is still failing, and after a bit of looking around at
other code I finally got hit with the clue hammer. Per port.h:

* On Windows, setvbuf() does not support _IOLBF mode, and interprets that
* as _IOFBF. To add insult to injury, setvbuf(file, NULL, _IOFBF, 0)
* crashes outright if "parameter validation" is enabled. Therefore, in
* places where we'd like to select line-buffered mode, we fall back to
* unbuffered mode instead on Windows. Always use PG_IOLBF not _IOLBF
* directly in order to implement this behavior.

You want to do the honors? And do something about that shift bug
while at it.

Ooh, wow ... now that is a silly bug! Thanks, I'll push the fix in a
minute.

Andrew also noted that the use of command_ok() in the test file eats all
the output and is what is preventing us from seeing it in the first.
I'll try to get rid of that together with the rest.

--
�lvaro Herrera Valdivia, Chile

#214'alvherre@alvh.no-ip.org'
alvherre@alvh.no-ip.org
In reply to: 'alvherre@alvh.no-ip.org' (#213)
Re: libpq debug log

On 2021-Apr-01, 'alvherre@alvh.no-ip.org' wrote:

Ooh, wow ... now that is a silly bug! Thanks, I'll push the fix in a
minute.

It still didn't fix it! Drongo is now reporting a difference in the
expected trace -- and the differences all seem to be message lengths.
Now that is pretty mysterious, because the messages themselves are
printed identically. Perl's output is pretty unhelpful, but I wrote them to a
file and diffed manually; it's attached.

So for example when we expect
! # B 123 ErrorResponse S "ERROR" V "ERROR" C "42601" M "cannot insert multiple commands into a prepared statement" F "SSSS" L "SSSS" R "SSSS" \\x00

we actually get
! # B 173 ErrorResponse S "ERROR" V "ERROR" C "42601" M "cannot insert multiple commands into a prepared statement" F "SSSS" L "SSSS" R "SSSS" \\x00

[slaps forehead] I suppose the difference must be that the message
length includes the redacted string fields. So the file in the F file
is 50 chars longer, *kaboom*. (I'm actually very surprised that this
didn't blow up earlier.)

I'm not sure what to do about this. Maybe the message length for
ErrorResponse/NoticeResponse ought to be redacted too.

--
�lvaro Herrera 39�49'30"S 73�17'W

#215iwata.aya@fujitsu.com
iwata.aya@fujitsu.com
In reply to: 'alvherre@alvh.no-ip.org' (#214)
RE: libpq debug log

Hi Alvaro san

Thank you for your fix of trace log code.

From: 'alvherre@alvh.no-ip.org' <alvherre@alvh.no-ip.org>
Sent: Friday, April 2, 2021 11:30 AM

...

It still didn't fix it! Drongo is now reporting a difference in the expected trace --
and the differences all seem to be message lengths.
Now that is pretty mysterious, because the messages themselves are printed
identically. Perl's output is pretty unhelpful, but I wrote them to a file and diffed
manually; it's attached.

Do ErrorResponse and NoticeResponse vary from test to test ...?
If so, it seemed difficult to make the trace logs available for regression test.
I will also investigate the cause of this issue.

Regards,
Aya Iwata

#216Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: iwata.aya@fujitsu.com (#215)
Re: libpq debug log

At Fri, 2 Apr 2021 02:56:44 +0000, "iwata.aya@fujitsu.com" <iwata.aya@fujitsu.com> wrote in

Hi Alvaro san

Thank you for your fix of trace log code.

From: 'alvherre@alvh.no-ip.org' <alvherre@alvh.no-ip.org>
Sent: Friday, April 2, 2021 11:30 AM

...

It still didn't fix it! Drongo is now reporting a difference in the expected trace --
and the differences all seem to be message lengths.
Now that is pretty mysterious, because the messages themselves are printed
identically. Perl's output is pretty unhelpful, but I wrote them to a file and diffed
manually; it's attached.

Do ErrorResponse and NoticeResponse vary from test to test ...?
If so, it seemed difficult to make the trace logs available for regression test.
I will also investigate the cause of this issue.

The redacted fields, F, L and R contained source file, souce line and
source function respectively. It is reasonable guess that the
difference comes from them but I'm not sure how they make a difference
of 50 bytes in length...

Anyway if the length is wrong, we should get an error after emitting
the log line.

if (logCursor - 1 != length)
fprintf(conn->Pfdebug,
"mismatched message length: consumed %d, expected %d\n",
logCursor - 1, length);

So, the cheapest measure for regression test would be just making the
length field, while regress is true...

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#217Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kyotaro Horiguchi (#216)
Re: libpq debug log

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

At Fri, 2 Apr 2021 02:56:44 +0000, "iwata.aya@fujitsu.com" <iwata.aya@fujitsu.com> wrote in

Do ErrorResponse and NoticeResponse vary from test to test ...?
If so, it seemed difficult to make the trace logs available for regression test.
I will also investigate the cause of this issue.

The redacted fields, F, L and R contained source file, souce line and
source function respectively. It is reasonable guess that the
difference comes from them but I'm not sure how they make a difference
of 50 bytes in length...

Some compilers include the full path of the source file in F ...

regards, tom lane

#218Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Kyotaro Horiguchi (#216)
Re: libpq debug log

At Fri, 02 Apr 2021 14:40:09 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

At Fri, 2 Apr 2021 02:56:44 +0000, "iwata.aya@fujitsu.com" <iwata.aya@fujitsu.com> wrote in

Hi Alvaro san

Thank you for your fix of trace log code.

From: 'alvherre@alvh.no-ip.org' <alvherre@alvh.no-ip.org>
Sent: Friday, April 2, 2021 11:30 AM

...

It still didn't fix it! Drongo is now reporting a difference in the expected trace --
and the differences all seem to be message lengths.
Now that is pretty mysterious, because the messages themselves are printed
identically. Perl's output is pretty unhelpful, but I wrote them to a file and diffed
manually; it's attached.

Do ErrorResponse and NoticeResponse vary from test to test ...?
If so, it seemed difficult to make the trace logs available for regression test.
I will also investigate the cause of this issue.

The redacted fields, F, L and R contained source file, souce line and
source function respectively. It is reasonable guess that the
difference comes from them but I'm not sure how they make a difference
of 50 bytes in length...

Anyway if the length is wrong, we should get an error after emitting
the log line.

if (logCursor - 1 != length)
fprintf(conn->Pfdebug,
"mismatched message length: consumed %d, expected %d\n",
logCursor - 1, length);

So, the cheapest measure for regression test would be just making the

So, the cheapest measure for regression test would be just *masking* the

length field, while regress is true...

tired?

--
Kyotaro Horiguchi
NTT Open Source Software Center

#219Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Tom Lane (#217)
Re: libpq debug log

At Fri, 02 Apr 2021 01:45:06 -0400, Tom Lane <tgl@sss.pgh.pa.us> wrote in

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

At Fri, 2 Apr 2021 02:56:44 +0000, "iwata.aya@fujitsu.com" <iwata.aya@fujitsu.com> wrote in

Do ErrorResponse and NoticeResponse vary from test to test ...?
If so, it seemed difficult to make the trace logs available for regression test.
I will also investigate the cause of this issue.

The redacted fields, F, L and R contained source file, souce line and
source function respectively. It is reasonable guess that the
difference comes from them but I'm not sure how they make a difference
of 50 bytes in length...

Some compilers include the full path of the source file in F ...

Ah. I suspected that but dismissed the possibility looking that on my
environment, gcc8. That completely makes sense. Thank you!

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#220Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Kyotaro Horiguchi (#218)
Re: libpq debug log

On 2021-Apr-02, Kyotaro Horiguchi wrote:

At Fri, 02 Apr 2021 14:40:09 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

So, the cheapest measure for regression test would be just making the

So, the cheapest measure for regression test would be just *masking* the

length field, while regress is true...

Yeah.

tired?

Same here.

--
�lvaro Herrera 39�49'30"S 73�17'W
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end." (2nd Commandment for C programmers)

#221Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Alvaro Herrera (#220)
1 attachment(s)
Re: libpq debug log

On 2021-Apr-02, Alvaro Herrera wrote:

On 2021-Apr-02, Kyotaro Horiguchi wrote:

At Fri, 02 Apr 2021 14:40:09 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in

So, the cheapest measure for regression test would be just making the

So, the cheapest measure for regression test would be just *masking* the

length field, while regress is true...

Yeah.

As in the attached patch.

--
�lvaro Herrera Valdivia, Chile
"Investigaci�n es lo que hago cuando no s� lo que estoy haciendo"
(Wernher von Braun)

Attachments:

0001-Suppress-length-of-Notice-Error-in-PQtrace-regress-m.patchtext/x-diff; charset=us-asciiDownload
From 387a15ec80cfcdd2279b109215b9f658bc972748 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 2 Apr 2021 10:51:42 -0300
Subject: [PATCH] Suppress length of Notice/Error in PQtrace regress mode

---
 src/interfaces/libpq/fe-trace.c                          | 9 ++++++++-
 .../modules/libpq_pipeline/traces/pipeline_abort.trace   | 8 ++++----
 src/test/modules/libpq_pipeline/traces/transaction.trace | 6 +++---
 3 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/src/interfaces/libpq/fe-trace.c b/src/interfaces/libpq/fe-trace.c
index 9a4595f5c8..298713c602 100644
--- a/src/interfaces/libpq/fe-trace.c
+++ b/src/interfaces/libpq/fe-trace.c
@@ -540,7 +540,14 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	length = (int) pg_ntoh32(length);
 	logCursor += 4;
 
-	fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
+	/*
+	 * In regress mode, suppress the length of ErrorResponse and
+	 * NoticeResponse -- they vary depending on compiler.
+	 */
+	if (regress && !toServer && (id == 'E' || id == 'N'))
+		fprintf(conn->Pfdebug, "%s\tNN\t", prefix);
+	else
+		fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
 
 	switch (id)
 	{
diff --git a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
index b061435785..254e485997 100644
--- a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
+++ b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
@@ -1,5 +1,5 @@
 F	42	Query	 "DROP TABLE IF EXISTS pq_pipeline_demo"
-B	123	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_demo" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_demo" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
 B	15	CommandComplete	 "DROP TABLE"
 B	5	ReadyForQuery	 I
 F	99	Query	 "CREATE UNLOGGED TABLE pq_pipeline_demo(id serial primary key, itemno integer,int8filler int8);"
@@ -27,7 +27,7 @@ B	4	ParseComplete
 B	4	BindComplete
 B	4	NoData
 B	15	CommandComplete	 "INSERT 0 1"
-B	217	ErrorResponse	 S "ERROR" V "ERROR" C "42883" M "function no_such_function(integer) does not exist" H "No function matches the given name and argument types. You might need to add explicit type casts." P "8" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "42883" M "function no_such_function(integer) does not exist" H "No function matches the given name and argument types. You might need to add explicit type casts." P "8" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 I
 B	4	ParseComplete
 B	4	BindComplete
@@ -39,7 +39,7 @@ F	12	Bind	 "" "" 0 0 0
 F	6	Describe	 P ""
 F	9	Execute	 "" 0
 F	4	Sync
-B	123	ErrorResponse	 S "ERROR" V "ERROR" C "42601" M "cannot insert multiple commands into a prepared statement" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "42601" M "cannot insert multiple commands into a prepared statement" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 I
 F	54	Parse	 "" "SELECT 1.0/g FROM generate_series(3, -1, -1) g" 0
 F	12	Bind	 "" "" 0 0 0
@@ -52,7 +52,7 @@ B	33	RowDescription	 1 "?column?" NNNN 0 NNNN 65535 -1 0
 B	32	DataRow	 1 22 '0.33333333333333333333'
 B	32	DataRow	 1 22 '0.50000000000000000000'
 B	32	DataRow	 1 22 '1.00000000000000000000'
-B	70	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 I
 F	40	Query	 "SELECT itemno FROM pq_pipeline_demo"
 B	31	RowDescription	 1 "itemno" NNNN 2 NNNN 4 -1 0
diff --git a/src/test/modules/libpq_pipeline/traces/transaction.trace b/src/test/modules/libpq_pipeline/traces/transaction.trace
index 0267a534fa..1dcc2373c0 100644
--- a/src/test/modules/libpq_pipeline/traces/transaction.trace
+++ b/src/test/modules/libpq_pipeline/traces/transaction.trace
@@ -1,5 +1,5 @@
 F	79	Query	 "DROP TABLE IF EXISTS pq_pipeline_tst;CREATE TABLE pq_pipeline_tst (id int)"
-B	122	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_tst" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_tst" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
 B	15	CommandComplete	 "DROP TABLE"
 B	17	CommandComplete	 "CREATE TABLE"
 B	5	ReadyForQuery	 I
@@ -40,9 +40,9 @@ B	4	BindComplete
 B	4	NoData
 B	10	CommandComplete	 "BEGIN"
 B	4	ParseComplete
-B	65	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 E
-B	145	ErrorResponse	 S "ERROR" V "ERROR" C "25P02" M "current transaction is aborted, commands ignored until end of transaction block" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "25P02" M "current transaction is aborted, commands ignored until end of transaction block" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 E
 B	4	BindComplete
 B	4	NoData
-- 
2.20.1

#222Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#221)
Re: libpq debug log

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

As in the attached patch.

+1, but the comment could be more specific. Maybe like "In regress mode,
suppress the length of ErrorResponse and NoticeResponse messages --- the F
(file name) field, in particular, can vary in length depending on compiler
used."

Actually though ... I recall now that elog.c tries to standardize the F
output by stripping the path part:

/* keep only base name, useful especially for vpath builds */
slash = strrchr(filename, '/');
if (slash)
filename = slash + 1;

I bet what is happening on drongo is that the compiler has generated a
__FILE__ value that contains backslashes not slashes, and this code
doesn't know how to shorten those. So maybe instead of lobotomizing
this test, we should fix that.

slash = strrchr(filename, '/');
+ if (!slash)
+ slash = strrchr(filename, '\\');
if (slash)
filename = slash + 1;

regards, tom lane

#223Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#222)
Re: libpq debug log

I wrote:

I bet what is happening on drongo is that the compiler has generated a
__FILE__ value that contains backslashes not slashes, and this code
doesn't know how to shorten those. So maybe instead of lobotomizing
this test, we should fix that.

Did that, but we'll have to wait a few hours to see if it makes
drongo happy.

regards, tom lane

#224Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#223)
Re: libpq debug log

On 2021-Apr-02, Tom Lane wrote:

I wrote:

I bet what is happening on drongo is that the compiler has generated a
__FILE__ value that contains backslashes not slashes, and this code
doesn't know how to shorten those. So maybe instead of lobotomizing
this test, we should fix that.

Did that, but we'll have to wait a few hours to see if it makes
drongo happy.

Thanks, I really hope it does!

--
�lvaro Herrera Valdivia, Chile

#225Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#223)
Re: libpq debug log

I wrote:

I bet what is happening on drongo is that the compiler has generated a
__FILE__ value that contains backslashes not slashes, and this code
doesn't know how to shorten those. So maybe instead of lobotomizing
this test, we should fix that.

Did that, but we'll have to wait a few hours to see if it makes
drongo happy.

On third thought, maybe we should push your patch too. Although I think
53aafdb9f is going to fix the platform-specific aspect of this, we are
still going to risk some implementation dependence of the libpq_pipeline
results:

* Every so often, the number of digits in the reported line numbers
will change (999 -> 1001 or the like), due to changes in unrelated
code.

* Occasionally we refactor things so that the "same" error is reported
from a different file.

It's hard to judge whether that will happen often enough to be an
annoying maintenance problem, but there's definitely a hazard.
Not sure if we should proactively lobotomize the test, or wait to
see if we get annoyed.

In any case I'd like to wait till after drongo's next run, so
we can confirm whether or not the backslashes-in-__FILE__ hypothesis
is correct. If it is, then 53aafdb9f is a good fix on its own
merits, independently of libpq_pipeline.

regards, tom lane

#226Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#225)
Re: libpq debug log

On 2021-Apr-02, Tom Lane wrote:

On third thought, maybe we should push your patch too. Although I think
53aafdb9f is going to fix the platform-specific aspect of this, we are
still going to risk some implementation dependence of the libpq_pipeline
results:

* Every so often, the number of digits in the reported line numbers
will change (999 -> 1001 or the like), due to changes in unrelated
code.

Bah -- of course.

* Occasionally we refactor things so that the "same" error is reported
from a different file.

True. (This was the reason for masking F and R, but I didn't realize
that it'd have an effect in the message length).

It's hard to judge whether that will happen often enough to be an
annoying maintenance problem, but there's definitely a hazard.
Not sure if we should proactively lobotomize the test, or wait to
see if we get annoyed.

I suspect we're going to see enough bf failures if we don't suppress it,
because the problem is going to only show up with TAP testing enabled
and the src/test/modules tests, which perhaps not all committers run.

In any case I'd like to wait till after drongo's next run, so
we can confirm whether or not the backslashes-in-__FILE__ hypothesis
is correct. If it is, then 53aafdb9f is a good fix on its own
merits, independently of libpq_pipeline.

Wilco.

--
�lvaro Herrera Valdivia, Chile
Essentially, you're proposing Kevlar shoes as a solution for the problem
that you want to walk around carrying a loaded gun aimed at your foot.
(Tom Lane)

#227Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#222)
1 attachment(s)
Re: libpq debug log

On 2021-Apr-02, Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

As in the attached patch.

+1, but the comment could be more specific. Maybe like "In regress mode,
suppress the length of ErrorResponse and NoticeResponse messages --- the F
(file name) field, in particular, can vary in length depending on compiler
used."

With your commit this is no longer true, so how about the attached?

--
�lvaro Herrera Valdivia, Chile

Attachments:

v2-0001-Suppress-length-of-Notice-Error-in-PQtrace-regres.patchtext/x-diff; charset=us-asciiDownload
From 80af8f687ea3665a045d9005e0ec1fc53e1a393a Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Fri, 2 Apr 2021 10:51:42 -0300
Subject: [PATCH v2] Suppress length of Notice/Error in PQtrace regress mode

A (relatively minor) annoyance of ErrorResponse/NoticeResponse messages
as printed by PQtrace() is that their length might vary when we move
error messages from one file to another, one function to another, or
even when their location line number changes number of digits.

To avoid having to adjust expected files for some tests, make the
regress mode of PQtrace() suppress the length word of those messages.

Discussion: https://postgr.es/m/20210402023010.GA13563@alvherre.pgsql
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
---
 src/interfaces/libpq/fe-trace.c                       | 11 ++++++++++-
 .../libpq_pipeline/traces/pipeline_abort.trace        |  8 ++++----
 .../modules/libpq_pipeline/traces/transaction.trace   |  6 +++---
 3 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/src/interfaces/libpq/fe-trace.c b/src/interfaces/libpq/fe-trace.c
index 9a4595f5c8..ca24869e91 100644
--- a/src/interfaces/libpq/fe-trace.c
+++ b/src/interfaces/libpq/fe-trace.c
@@ -540,7 +540,16 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer)
 	length = (int) pg_ntoh32(length);
 	logCursor += 4;
 
-	fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
+	/*
+	 * In regress mode, suppress the length of ErrorResponse and
+	 * NoticeResponse.  The F (file name), L (line number) and R (routine
+	 * name) fields can change as server code is modified, and if their
+	 * lengths differ from the originals, that would break tests.
+	 */
+	if (regress && !toServer && (id == 'E' || id == 'N'))
+		fprintf(conn->Pfdebug, "%s\tNN\t", prefix);
+	else
+		fprintf(conn->Pfdebug, "%s\t%d\t", prefix, length);
 
 	switch (id)
 	{
diff --git a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
index b061435785..254e485997 100644
--- a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
+++ b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
@@ -1,5 +1,5 @@
 F	42	Query	 "DROP TABLE IF EXISTS pq_pipeline_demo"
-B	123	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_demo" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_demo" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
 B	15	CommandComplete	 "DROP TABLE"
 B	5	ReadyForQuery	 I
 F	99	Query	 "CREATE UNLOGGED TABLE pq_pipeline_demo(id serial primary key, itemno integer,int8filler int8);"
@@ -27,7 +27,7 @@ B	4	ParseComplete
 B	4	BindComplete
 B	4	NoData
 B	15	CommandComplete	 "INSERT 0 1"
-B	217	ErrorResponse	 S "ERROR" V "ERROR" C "42883" M "function no_such_function(integer) does not exist" H "No function matches the given name and argument types. You might need to add explicit type casts." P "8" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "42883" M "function no_such_function(integer) does not exist" H "No function matches the given name and argument types. You might need to add explicit type casts." P "8" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 I
 B	4	ParseComplete
 B	4	BindComplete
@@ -39,7 +39,7 @@ F	12	Bind	 "" "" 0 0 0
 F	6	Describe	 P ""
 F	9	Execute	 "" 0
 F	4	Sync
-B	123	ErrorResponse	 S "ERROR" V "ERROR" C "42601" M "cannot insert multiple commands into a prepared statement" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "42601" M "cannot insert multiple commands into a prepared statement" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 I
 F	54	Parse	 "" "SELECT 1.0/g FROM generate_series(3, -1, -1) g" 0
 F	12	Bind	 "" "" 0 0 0
@@ -52,7 +52,7 @@ B	33	RowDescription	 1 "?column?" NNNN 0 NNNN 65535 -1 0
 B	32	DataRow	 1 22 '0.33333333333333333333'
 B	32	DataRow	 1 22 '0.50000000000000000000'
 B	32	DataRow	 1 22 '1.00000000000000000000'
-B	70	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 I
 F	40	Query	 "SELECT itemno FROM pq_pipeline_demo"
 B	31	RowDescription	 1 "itemno" NNNN 2 NNNN 4 -1 0
diff --git a/src/test/modules/libpq_pipeline/traces/transaction.trace b/src/test/modules/libpq_pipeline/traces/transaction.trace
index 0267a534fa..1dcc2373c0 100644
--- a/src/test/modules/libpq_pipeline/traces/transaction.trace
+++ b/src/test/modules/libpq_pipeline/traces/transaction.trace
@@ -1,5 +1,5 @@
 F	79	Query	 "DROP TABLE IF EXISTS pq_pipeline_tst;CREATE TABLE pq_pipeline_tst (id int)"
-B	122	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_tst" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	NoticeResponse	 S "NOTICE" V "NOTICE" C "00000" M "table "pq_pipeline_tst" does not exist, skipping" F "SSSS" L "SSSS" R "SSSS" \x00
 B	15	CommandComplete	 "DROP TABLE"
 B	17	CommandComplete	 "CREATE TABLE"
 B	5	ReadyForQuery	 I
@@ -40,9 +40,9 @@ B	4	BindComplete
 B	4	NoData
 B	10	CommandComplete	 "BEGIN"
 B	4	ParseComplete
-B	65	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 E
-B	145	ErrorResponse	 S "ERROR" V "ERROR" C "25P02" M "current transaction is aborted, commands ignored until end of transaction block" F "SSSS" L "SSSS" R "SSSS" \x00
+B	NN	ErrorResponse	 S "ERROR" V "ERROR" C "25P02" M "current transaction is aborted, commands ignored until end of transaction block" F "SSSS" L "SSSS" R "SSSS" \x00
 B	5	ReadyForQuery	 E
 B	4	BindComplete
 B	4	NoData
-- 
2.20.1

#228Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#227)
Re: libpq debug log

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

On 2021-Apr-02, Tom Lane wrote:

+1, but the comment could be more specific. Maybe like "In regress mode,
suppress the length of ErrorResponse and NoticeResponse messages --- the F
(file name) field, in particular, can vary in length depending on compiler
used."

With your commit this is no longer true, so how about the attached?

Right. Your text looks fine.

(I see drongo is now green, so it looks like its compiler does indeed emit
backslash-separated full paths.)

regards, tom lane

#229Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#228)
Re: libpq debug log

On 2021-Apr-02, Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

On 2021-Apr-02, Tom Lane wrote:

+1, but the comment could be more specific. Maybe like "In regress mode,
suppress the length of ErrorResponse and NoticeResponse messages --- the F
(file name) field, in particular, can vary in length depending on compiler
used."

With your commit this is no longer true, so how about the attached?

Right. Your text looks fine.

Pushed now.

--
�lvaro Herrera 39�49'30"S 73�17'W
"I'm always right, but sometimes I'm more right than other times."
(Linus Torvalds)

#230Noah Misch
noah@leadboat.com
In reply to: Alvaro Herrera (#229)
Re: libpq debug log

On Fri, Apr 09, 2021 at 05:16:11PM -0400, Alvaro Herrera wrote:

Pushed now.

This added a PQtraceSetFlags() function. We have a dozen PQset*() functions,
but this and PQresultSetInstanceData() are the only PQSomethingSet()
functions. Is it okay if I rename it to PQsetTraceFlags()? I think that
would be more idiomatic for libpq.

#231Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Noah Misch (#230)
Re: libpq debug log

On 2021-Jun-04, Noah Misch wrote:

On Fri, Apr 09, 2021 at 05:16:11PM -0400, Alvaro Herrera wrote:

Pushed now.

This added a PQtraceSetFlags() function. We have a dozen PQset*() functions,
but this and PQresultSetInstanceData() are the only PQSomethingSet()
functions. Is it okay if I rename it to PQsetTraceFlags()? I think that
would be more idiomatic for libpq.

Hmm, there is an obvious parallel between PQtrace() and
PQtraceSetFlags() which is lost with your proposed rename. I'm not
wedded to the name though, so I'm just -0.1 on the rename. If you feel
strongly about it, I won't oppose it.

--
�lvaro Herrera Valdivia, Chile
"La espina, desde que nace, ya pincha" (Proverbio africano)

#232Robert Haas
robertmhaas@gmail.com
In reply to: Alvaro Herrera (#231)
Re: libpq debug log

On Sat, Jun 5, 2021 at 11:03 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

This added a PQtraceSetFlags() function. We have a dozen PQset*() functions,
but this and PQresultSetInstanceData() are the only PQSomethingSet()
functions. Is it okay if I rename it to PQsetTraceFlags()? I think that
would be more idiomatic for libpq.

Hmm, there is an obvious parallel between PQtrace() and
PQtraceSetFlags() which is lost with your proposed rename. I'm not
wedded to the name though, so I'm just -0.1 on the rename. If you feel
strongly about it, I won't oppose it.

I'm +1 for the rename. I think it's a lot more idiomatic.

--
Robert Haas
EDB: http://www.enterprisedb.com

#233Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#232)
Re: libpq debug log

On Mon, Jun 07, 2021 at 11:37:58AM -0400, Robert Haas wrote:

On Sat, Jun 5, 2021 at 11:03 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

This added a PQtraceSetFlags() function. We have a dozen PQset*() functions,
but this and PQresultSetInstanceData() are the only PQSomethingSet()
functions. Is it okay if I rename it to PQsetTraceFlags()? I think that
would be more idiomatic for libpq.

Hmm, there is an obvious parallel between PQtrace() and
PQtraceSetFlags() which is lost with your proposed rename. I'm not
wedded to the name though, so I'm just -0.1 on the rename. If you feel
strongly about it, I won't oppose it.

I'm +1 for the rename. I think it's a lot more idiomatic.

At +1 vs. -0.1, I would have withdrawn the proposal. Seeing +2 vs. -0.1, I
have pushed it.