Client Connection redirection support for PostgreSQL

Started by Satyanarayana Narlapuramabout 8 years ago15 messages
#1Satyanarayana Narlapuram
Satyanarayana.Narlapuram@microsoft.com

Proposal:
Add the ability to the PostgreSQL server instance to route the traffic to a different server instance based on the rules defined in server's pg_bha.conf configuration file. At a high level this enables offloading the user requests to a different server instance based on the rules defined in the pg_hba.conf configuration file. Some of the interesting scenarios this enables include but not limited to - rerouting traffic based on the client hosts, users, database, etc. specified, redirecting read-only query traffic to the hot stand by replicas, and in multi-master scenarios.
The rules to route the traffic will be provided in the pg_hba.conf file. The proposal is to add a new optional field 'RoutingList' to the record format. The RoutingList contains comma-seperated list of one or more servers that can be routed the traffic to. In the absence of this new field there is no change to the current login code path for both the server and the client. RoutingList can be updated for each new connection to balance the load across multiple server instances
RoutingList format:
server_address1:port, server_address2:port...
The message flow

1. Client connects to the server, and server accepts the connections
2. Client sends the startup message
3. Server looks at the rules configured in the pg_hba.conf file and
* If the rule matches redirection

i. Send a special message with the RoutingList described above

ii. Server disconnects

* If the rule doesn't have RoutingList defined

i. Server proceeds in the existing code path and sends auth request

1. Client gets the list of addresses and attempts to connect to a server in the list provided until the first successful connections is established or the list is exhausted. If the client can't connect to any server instance on the RoutingList, client reports the login failure message.

Backward compatibility:
There are a few ways to provide the backward compatibility, and each approach has their own advantages and disadvantage and are listed below

1. Bumping the protocol version - old server instances may not understand the new client protocol
2. Adding additional optional parameter routing_enabled, without bumping the protocol version. In this approach, old Postgres server instances may not understand this and fail the connections.
3. The current proposal - to keep it in the hba.conf and let the server admin deal with the configuration by taking conscious choice on the configuration of routing list based on the clients connecting to the server instance.
Backward compatibility scenarios:

* The feature is not usable for the existing clients, and the new servers shouldn't set the routing list if they expect any connections from the legacy clients. We should do either (1) or (2) in the above list to achieve this. Otherwise need to rely on the admin to take care of the settings.
* For the new client connecting to the old server, there is no change in the message flow
* For the new clients to the new server, the message flow will be based on the routing list filed in the configuration.
This proposal is in very early stage, comments and feedback is very much appreciated.
Thanks,
Satya

#2David Fetter
david@fetter.org
In reply to: Satyanarayana Narlapuram (#1)
Re: Client Connection redirection support for PostgreSQL

On Thu, Nov 02, 2017 at 06:02:43AM +0000, Satyanarayana Narlapuram wrote:

Proposal:
Add the ability to the PostgreSQL server instance to route the
traffic to a different server instance based on the rules defined in
server's pg_bha.conf configuration file. At a high level this
enables offloading the user requests to a different server instance
based on the rules defined in the pg_hba.conf configuration file.
Some of the interesting scenarios this enables include but not
limited to - rerouting traffic based on the client hosts, users,
database, etc. specified, redirecting read-only query traffic to the
hot stand by replicas, and in multi-master scenarios.

What advantages do you see in doing this in the backend over the
current system where the concerns are separated, i.e. people use
connection poolers like pgbouncer to do the routing?

The rules to route the traffic will be provided in the pg_hba.conf
file. The proposal is to add a new optional field 'RoutingList' to
the record format. The RoutingList contains comma-seperated list of
one or more servers that can be routed the traffic to. In the
absence of this new field there is no change to the current login
code path for both the server and the client. RoutingList can be
updated for each new connection to balance the load across multiple
server instances

RoutingList format:
server_address1:port, server_address2:port...

Would it make sense also to include an optional routing algorithm or
pointer to a routing function for each RoutingList, or do you see this
as entirely the client's responsibility?

The message flow

1. Client connects to the server, and server accepts the connections

How does this work with SSL?

2. Client sends the startup message
3. Server looks at the rules configured in the pg_hba.conf file and
* If the rule matches redirection
i. Send a special message with the RoutingList described above
ii. Server disconnects

* If the rule doesn't have RoutingList defined

i. Server proceeds in the existing code path and sends auth request

1. Client gets the list of addresses and attempts to connect to a
server in the list provided until the first successful connections
is established or the list is exhausted. If the client can't
connect to any server instance on the RoutingList, client reports
the login failure message.

Backward compatibility:
There are a few ways to provide the backward compatibility, and each
approach has their own advantages and disadvantage and are listed
below

1. Bumping the protocol version - old server instances may not
understand the new client protocol

This sounds more attractive, assuming that the feature is.

2. Adding additional optional parameter routing_enabled, without
bumping the protocol version. In this approach, old Postgres
server instances may not understand this and fail the connections.

Silently changing an API without bumping the protocol version seems
like a bad plan, even when it's an additive change as you propose here.

3. The current proposal - to keep it in the hba.conf and let the
server admin deal with the configuration by taking conscious
choice on the configuration of routing list based on the clients
connecting to the server instance.

How would clients identify themselves as eligible without a protocol
version bump?

Backward compatibility scenarios:

* The feature is not usable for the existing clients, and the
new servers shouldn't set the routing list if they expect any
connections from the legacy clients. We should do either (1) or
(2) in the above list to achieve this. Otherwise need to rely on
the admin to take care of the settings.
* For the new client connecting to the old server, there is no
change in the message flow

So to DoS the server, what's required is a flock of old clients? I
presume there's a good reason to reroute rather than serve these
requests.

* For the new clients to the new server, the message flow will be based on the routing list filed in the configuration.
This proposal is in very early stage, comments and feedback is very much appreciated.

Comments and feedback have begun.

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Craig Ringer
craig@2ndquadrant.com
In reply to: Satyanarayana Narlapuram (#1)
Re: Client Connection redirection support for PostgreSQL

On 2 November 2017 at 14:02, Satyanarayana Narlapuram
<Satyanarayana.Narlapuram@microsoft.com> wrote:

Proposal:

Add the ability to the PostgreSQL server instance to route the traffic to a
different server instance based on the rules defined in server’s pg_bha.conf
configuration file. At a high level this enables offloading the user
requests to a different server instance based on the rules defined in the
pg_hba.conf configuration file.

pg_hba.conf is "host based access [control]" . I'm not sure it's
really the right place.

Some of the interesting scenarios this
enables include but not limited to - rerouting traffic based on the client
hosts, users, database, etc. specified, redirecting read-only query traffic
to the hot stand by replicas, and in multi-master scenarios.

When this has come up before, one of the issues has been determining
what exactly should constitute "read only" vs "read write" for the
purposes of redirecting work.

There are a bunch of issues there. If you're doing "read only" txns
and then do something "read write" and get redirected, the destination
doesn't have your prepared statements, any WITH HOLD cursors, temp
tables, etc you were working with. Strangeness ensues.

But we now have a session-intent stuff though. So we could possibly do
it at session level.

Backends used just for a redirect would be pretty expensive though.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Satyanarayana Narlapuram
Satyanarayana.Narlapuram@microsoft.com
In reply to: David Fetter (#2)
Re: Client Connection redirection support for PostgreSQL

What advantages do you see in doing this in the backend over the current system where the concerns are separated, i.e. people use connection poolers like pgbouncer to do the routing?

IMHO connection pooler is not great for latency sensitive applications. For small deployments, proxy is an overhead. For example, in the cloud environment, the proxy has to sit in one data center / region and has to server the client requests serving from other data centers.

Would it make sense also to include an optional routing algorithm or pointer to a routing function for each RoutingList, or do you see this as entirely the client's responsibility?

This is a great point, I haven't put much though into this beyond round robin / random shuffling. Providing the priority list of endpoints to the client from the server will allow client connections balanced accordingly. However, it is up to the client implementation to honor the list.

How does this work with SSL?

The protocol doesn't change much with SSL, and after the handshake, startup message is sent to the server from the client, and the new message flow kicks on the server based on the routing list.

1. Bumping the protocol version - old server instances may not understand the new client protocol

This sounds more attractive, assuming that the feature is.

I agree, bumping the protocol version makes things simple.

3. The current proposal - to keep it in the hba.conf and let the
server admin deal with the configuration by taking conscious
choice on the configuration of routing list based on the clients
connecting to the server instance.

How would clients identify themselves as eligible without a protocol version bump?

Either through optional parameter, or controlled configuration by the server administrator are the only choices.
Protocol bump seems to me is a better idea here.

So to DoS the server, what's required is a flock of old clients? I presume there's a good reason to reroute rather than serve these requests.

Possible, but I would say the server admin understands where the requests are coming from (old / new client) and does the capacity planning accordingly.

Comments and feedback have begun.

Thank you :)

Thanks,
Satya

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Satyanarayana Narlapuram
Satyanarayana.Narlapuram@microsoft.com
In reply to: Craig Ringer (#3)
Re: Client Connection redirection support for PostgreSQL

pg_hba.conf is "host based access [control]" . I'm not sure it's really the right place.

I am open to have another configuration file, say routing_list.conf to define the routing rules, but felt it is easy to extend the hba conf file.

But we now have a session-intent stuff though. So we could possibly do it at session level.

Session intent can be used as an obvious hint for the routing to kick in. This can be a rule in the routing list to route the read intent sessions round robin across multiple secondary replicas.

Backends used just for a redirect would be pretty expensive though.

It is somewhat expensive as the new process fork has to happen for each new connection. The advantage is that it makes proxies optional (if the middle tier can do connection management), and all the routing configurations can be within the server.
This also benefits latency sensitive applications not going through proxy.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Robert Haas
robertmhaas@gmail.com
In reply to: Craig Ringer (#3)
Re: Client Connection redirection support for PostgreSQL

On Thu, Nov 2, 2017 at 4:33 PM, Craig Ringer <craig@2ndquadrant.com> wrote:

Add the ability to the PostgreSQL server instance to route the traffic to a
different server instance based on the rules defined in server’s pg_bha.conf
configuration file. At a high level this enables offloading the user
requests to a different server instance based on the rules defined in the
pg_hba.conf configuration file.

pg_hba.conf is "host based access [control]" . I'm not sure it's
really the right place.

Well, we could invent someplace else, but I'm not sure I quite see the
point (full disclosure: I suggested the idea of doing this via
pg_hba.conf in an off-list discussion).

I do think the functionality is useful, for the same reasons that HTTP
redirects are useful. For example, let's say you have all of your
databases for various clients on a single instance. Then, one client
starts using a lot more resources, so you want to move that client to
a separate instance on another VM. You can set up logical replication
to replicate all of the data to the new instance, and then add a
pg_hba.conf entry to redirect connections to that database to the new
master (this would be even smoother if we had multi-master replication
in core). So now that client is moved off to another machine in a
completely client-transparent way. I think that's pretty cool.

When this has come up before, one of the issues has been determining
what exactly should constitute "read only" vs "read write" for the
purposes of redirecting work.

Yes, that needs some thought.

Backends used just for a redirect would be pretty expensive though.

Not as expensive as proxying the whole connection, as pgpool and other
systems do today. I think the in-core use of this redirect
functionality is useful, but I think the real win would be optionally
using it in pgpool and pgbouncer.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Satyanarayana Narlapuram
Satyanarayana.Narlapuram@microsoft.com
In reply to: Robert Haas (#6)
3 attachment(s)
RE: [HACKERS] Client Connection redirection support for PostgreSQL

I simplified the patch and for now just allowed one server. Please find the attached patches, and the commit message.

Thanks,
Satya

-----Original Message-----
From: Robert Haas <robertmhaas@gmail.com>
Sent: Monday, November 6, 2017 5:56 AM
To: Craig Ringer <craig@2ndquadrant.com>
Cc: Satyanarayana Narlapuram <Satyanarayana.Narlapuram@microsoft.com>; PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Client Connection redirection support for PostgreSQL

On Thu, Nov 2, 2017 at 4:33 PM, Craig Ringer <craig@2ndquadrant.com> wrote:

Add the ability to the PostgreSQL server instance to route the
traffic to a different server instance based on the rules defined in
server’s pg_bha.conf configuration file. At a high level this enables
offloading the user requests to a different server instance based on
the rules defined in the pg_hba.conf configuration file.

pg_hba.conf is "host based access [control]" . I'm not sure it's
really the right place.

Well, we could invent someplace else, but I'm not sure I quite see the point (full disclosure: I suggested the idea of doing this via pg_hba.conf in an off-list discussion).

I do think the functionality is useful, for the same reasons that HTTP redirects are useful. For example, let's say you have all of your databases for various clients on a single instance. Then, one client starts using a lot more resources, so you want to move that client to a separate instance on another VM. You can set up logical replication to replicate all of the data to the new instance, and then add a pg_hba.conf entry to redirect connections to that database to the new master (this would be even smoother if we had multi-master replication in core). So now that client is moved off to another machine in a completely client-transparent way. I think that's pretty cool.

When this has come up before, one of the issues has been determining
what exactly should constitute "read only" vs "read write" for the
purposes of redirecting work.

Yes, that needs some thought.

Backends used just for a redirect would be pretty expensive though.

Not as expensive as proxying the whole connection, as pgpool and other systems do today. I think the in-core use of this redirect functionality is useful, but I think the real win would be optionally using it in pgpool and pgbouncer.

--
Robert Haas
EnterpriseDB: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.enterprisedb.com&amp;data=02%7C01%7CSatyanarayana.Narlapuram%40microsoft.com%7Caafef2039b194d9c02c308d5251e12bb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636455733453945798&amp;sdata=8qystAJQ6UhnB7WRQh5i4nF8cyBUvKc9QIBfy59y%2FX8%3D&amp;reserved=0
The Enterprise PostgreSQL Company

Attachments:

redirection_be.patchapplication/octet-stream; name=redirection_be.patchDownload
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 746d7cbb8a..82510fda63 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -206,6 +206,12 @@ static int pg_SSPI_make_upn(char *accountname,
 static int	CheckRADIUSAuth(Port *port);
 static int	PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
 
+/*----------------------------------------------------------------
+ * Connection Redirection
+ *----------------------------------------------------------------
+ */
+static int SendAlternativeServerName(Port *port, char **logdetail);
+
 
 /*
  * Maximum accepted size of GSS and SSPI authentication tokens.
@@ -598,6 +604,9 @@ ClientAuthentication(Port *port)
 		case uaTrust:
 			status = STATUS_OK;
 			break;
+		case uaRedirect:
+			status = SendAlternativeServerName(port, &logdetail);
+			break;
 	}
 
 	if (ClientAuthentication_hook)
@@ -609,6 +618,28 @@ ClientAuthentication(Port *port)
 		auth_failed(port, status, logdetail);
 }
 
+/*
+ * Send alternative server information packet to the frontend.
+ */
+static int
+SendAlternativeServerName(Port *port, char **logdetail)
+{
+	StringInfoData buf;
+
+	CHECK_FOR_INTERRUPTS();
+
+	pq_beginmessage(&buf, 'M');
+	pq_sendstring(&buf, "server");
+	pq_sendstring(&buf, port->hba->alternativeservername);
+	pq_sendstring(&buf, "port");
+	pq_sendstring(&buf, port->hba->alternativeserverport);
+
+	pq_endmessage(&buf);
+	pq_flush();
+	proc_exit(0);
+
+	return STATUS_OK;
+}
 
 /*
  * Send an authentication request packet to the frontend.
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index aa20f266b8..d211068f44 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -134,7 +134,8 @@ static const char *const UserAuthName[] =
 	"ldap",
 	"cert",
 	"radius",
-	"peer"
+	"peer",
+	"redirect"
 };
 
 
@@ -1358,6 +1359,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
 #endif
 	else if (strcmp(token->string, "radius") == 0)
 		parsedline->auth_method = uaRADIUS;
+	else if (strcmp(token->string, "redirect") == 0)
+		parsedline->auth_method = uaRedirect;
 	else
 	{
 		ereport(elevel,
@@ -1384,6 +1387,49 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
 		return NULL;
 	}
 
+	if (parsedline->auth_method == uaRedirect)
+	{
+		/* Get the alternative server name and port */
+		field = lnext(field);
+		if (!field)
+		{
+			ereport(elevel,
+				(errcode(ERRCODE_CONFIG_FILE_ERROR),
+				errmsg("end-of-line before alternative server name"),
+				errcontext("line %d of configuration file \"%s\"",
+				line_num, HbaFileName)));
+			*err_msg = "end-of-line before alternative server name";
+			return NULL;
+		}
+		tokens = lfirst(field);
+		if (tokens->length > 2)
+		{
+			ereport(elevel,
+				(errcode(ERRCODE_CONFIG_FILE_ERROR),
+				errmsg("multiple values specified for alternative server"),
+				errhint("Specify exactly one alternative server per line."),
+				errcontext("line %d of configuration file \"%s\"",
+				line_num, HbaFileName)));
+			*err_msg = "multiple values specified for alternative server";
+			return NULL;
+		}
+
+		tokencell = list_head(tokens);
+		token = lfirst(tokencell);
+		parsedline->alternativeservername = pstrdup(token->string);
+
+		if (tokens->length == 2)
+		{
+			tokencell = lnext(tokencell);
+			token = lfirst(tokencell);
+			parsedline->alternativeserverport = pstrdup(token->string);
+		}
+		else
+		{
+			pg_itoa(DEF_PGPORT, parsedline->alternativeserverport);
+		}
+	}
+
 	/*
 	 * XXX: When using ident on local connections, change it to peer, for
 	 * backwards compatibility.
@@ -2100,6 +2146,12 @@ check_hba(hbaPort *port)
 		if (!check_role(port->user_name, roleid, hba->roles))
 			continue;
 
+		/* Check the protocol version to see if the client supports redirection */
+		if (PG_PROTOCOL_MAJOR(port->proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST) ||
+			(PG_PROTOCOL_MAJOR(port->proto) == PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST) &&
+			 PG_PROTOCOL_MINOR(port->proto) < PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST)))
+			continue;
+
 		/* Found a record that matched! */
 		port->hba = hba;
 		return;
diff --git a/src/backend/libpq/pg_hba.conf.sample b/src/backend/libpq/pg_hba.conf.sample
index c853e36232..3af0972d48 100644
--- a/src/backend/libpq/pg_hba.conf.sample
+++ b/src/backend/libpq/pg_hba.conf.sample
@@ -43,7 +43,7 @@
 # directly connected to.
 #
 # METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
-# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert".
+# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius", "cert" or "redirect".
 # Note that "password" sends passwords in clear text; "md5" or
 # "scram-sha-256" are preferred since they send encrypted passwords.
 #
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 5f68f4c666..1e4f2bd6db 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -38,8 +38,9 @@ typedef enum UserAuth
 	uaLDAP,
 	uaCert,
 	uaRADIUS,
-	uaPeer
-#define USER_AUTH_LAST uaPeer	/* Must be last value of this enum */
+	uaPeer,
+	uaRedirect
+#define USER_AUTH_LAST uaRedirect	/* Must be last value of this enum */
 } UserAuth;
 
 typedef enum IPCompareMethod
@@ -99,6 +100,8 @@ typedef struct HbaLine
 	char	   *radiusidentifiers_s;
 	List	   *radiusports;
 	char	   *radiusports_s;
+	char	   *alternativeservername;
+	char	   *alternativeserverport;
 } HbaLine;
 
 typedef struct IdentLine
redirection_fe_doc.patchapplication/octet-stream; name=redirection_fe_doc.patchDownload
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index 53832d08e2..a98b2d0fcd 100644
--- a/doc/src/sgml/client-auth.sgml
+++ b/doc/src/sgml/client-auth.sgml
@@ -495,6 +495,18 @@ hostnossl  <replaceable>database</replaceable>  <replaceable>user</replaceable>
        </varlistentry>
 
        <varlistentry>
+        <term><literal>redirect</></term>
+        <listitem>
+         <para>
+          Redirects the connection to the alternative server specified if it
+          matches the requested database user name and IP address. This is
+          only available for Protocol Version 3.1 and beyond.
+          See <xref linkend="auth-redirect"> for details.
+         </para>
+        </listitem>
+       </varlistentry>
+
+       <varlistentry>
         <term><literal>ldap</literal></term>
         <listitem>
          <para>
@@ -624,7 +636,7 @@ hostnossl  <replaceable>database</replaceable>  <replaceable>user</replaceable>
    non-null <structfield>error</structfield> fields indicate problems in the
    corresponding lines of the file.
   </para>
- 
+
   <tip>
    <para>
     To connect to a particular database, a user must not only pass the
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 8d543334ae..529226c043 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -1809,7 +1809,7 @@ connectDBStart(PGconn *conn)
 	 */
 	conn->whichhost = 0;
 	conn->addr_cur = conn->connhost[0].addrlist;
-	conn->pversion = PG_PROTOCOL(3, 0);
+	conn->pversion = PG_PROTOCOL(3, 1);
 	conn->send_appname = true;
 	conn->status = CONNECTION_NEEDED;
 
@@ -2007,6 +2007,14 @@ PQconnectPoll(PGconn *conn)
 	int			optval;
 	PQExpBufferData savedMessage;
 
+	/* Variable declarations for redirection. */
+	int			originalMsgLen;		/* Length in bytes of message sans msg type */
+	int			runningMsgLen;		/* Length in bytes of message sans metadata */
+	int			availableMsgLen;
+	char		*altServer = NULL;
+	char		*altPort = NULL;
+	bool		redirectionError = false; /* Flag used to mark exceptions */
+
 	if (conn == NULL)
 		return PGRES_POLLING_FAILED;
 
@@ -2024,6 +2032,7 @@ PQconnectPoll(PGconn *conn)
 
 			/* These are reading states */
 		case CONNECTION_AWAITING_RESPONSE:
+		case CONNECTION_REDIRECTION:
 		case CONNECTION_AUTH_OK:
 			{
 				/* Load waiting data */
@@ -2655,6 +2664,12 @@ keep_going:						/* We will come back to here until there is
 					return PGRES_POLLING_READING;
 				}
 
+				if (beresp == 'M')
+				{
+					conn->status = CONNECTION_REDIRECTION;
+					goto keep_going;
+				}
+
 				/*
 				 * Validate message type: we expect only an authentication
 				 * request or an error here.  Anything else probably means
@@ -3097,6 +3112,125 @@ keep_going:						/* We will come back to here until there is
 				conn->status = CONNECTION_OK;
 				return PGRES_POLLING_OK;
 			}
+
+		case CONNECTION_REDIRECTION:
+			{
+				/* Mark 'M' consumed: a single byte for message type. */
+				conn->inCursor = conn->inStart + 1;
+
+				/* Obtain message length from packet. */
+				if (pqGetInt(&originalMsgLen, sizeof(int32), conn))
+					return PGRES_POLLING_READING;
+
+				/* Obtain the number of bytes in payload. */
+				runningMsgLen = originalMsgLen - sizeof(int32);
+
+				/* Enlarge buffer if payload's size is greater than what is available. */
+				availableMsgLen = conn->inEnd - conn->inCursor;
+				if (availableMsgLen < runningMsgLen)
+				{
+					if (pqCheckInBufferSpace(conn->inCursor + (size_t)runningMsgLen, conn))
+						return PGRES_POLLING_READING;
+				}
+
+				PQExpBuffer buf = createPQExpBuffer();
+				while (pqGets(buf, conn) != EOF)
+				{
+					if (!strcmp(buf->data, "server"))
+					{
+						if (pqGets(buf, conn) == EOF)
+						{
+							appendPQExpBuffer(&conn->errorMessage,
+								libpq_gettext("failed to obtain server value from redirection packet"));
+
+							redirectionError = true;
+							break;
+						}
+						altServer = strdup(buf->data);
+					}
+					else if (!strcmp(buf->data, "port"))
+					{
+						if (pqGets(buf, conn) == EOF)
+						{
+							appendPQExpBuffer(&conn->errorMessage,
+								libpq_gettext("failed to obtain port value from redirection packet"));
+
+							redirectionError = true;
+							break;
+						}
+						altPort = strdup(buf->data);
+					}
+					else
+					{
+						appendPQExpBuffer(&conn->errorMessage,
+							libpq_gettext("unknown key in redirection server packet"));
+						redirectionError = true;
+					}
+
+					if (redirectionError)
+					{
+						/* Free buffer to prevent memory leak on error. */
+						destroyPQExpBuffer(buf);
+
+						/* Free strdup'd variables. */
+						if (altServer)
+							free(altServer);
+
+						if (altPort)
+							free(altPort);
+
+						goto error_return;
+					}
+
+					/* Buffer length does not account for null-terminated strings. */
+					runningMsgLen -= buf->len + 1;
+				}
+
+				/* Free buffer used for reading string params in packet. */
+				destroyPQExpBuffer(buf);
+
+				/* Check for extraneous data in packet. */
+				if (conn->inCursor != conn->inStart + 1 + originalMsgLen)
+				{
+					appendPQExpBuffer(&conn->errorMessage,
+						libpq_gettext("Extraneous data in redirection packet from server\n"));
+
+					/* Free strdup'd variables. */
+					if (altServer)
+						free(altServer);
+
+					if (altPort)
+						free(altPort);
+
+					goto error_return;
+				}
+
+				/* Mark incoming data consumed */
+				conn->inStart = conn->inCursor;
+
+				/* Drop existing connection. */
+				pqDropConnection(conn, true);
+
+				/* Set connection parameters. */
+				if (conn->pghost)
+				{
+					free(conn->pghost);
+					conn->pghost = altServer;
+				}
+
+				if (conn->pgport)
+				{
+					free(conn->pgport);
+					conn->pgport = altPort;
+				}
+
+				/* connectDBStart() sets appropriate connection status. */
+				if (!connectOptions2(conn) || !connectDBStart(conn))
+					conn->status = CONNECTION_BAD;
+
+				goto keep_going;
+			}
+
 		case CONNECTION_CHECK_WRITABLE:
 			{
 				const char *displayed_host;
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index ed9c806861..2a8f036b4b 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -65,8 +65,9 @@ typedef enum
 	CONNECTION_NEEDED,			/* Internal state: connect() needed */
 	CONNECTION_CHECK_WRITABLE,	/* Check if we could make a writable
 								 * connection. */
-	CONNECTION_CONSUME			/* Wait for any pending message and consume
+	CONNECTION_CONSUME,			/* Wait for any pending message and consume
 								 * them. */
+	CONNECTION_REDIRECTION		/* Redirection */
 } ConnStatusType;
 
 typedef enum
commit_message.txttext/plain; name=commit_message.txtDownload
#8Robert Haas
robertmhaas@gmail.com
In reply to: Satyanarayana Narlapuram (#7)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

On Mon, Feb 12, 2018 at 1:56 PM, Satyanarayana Narlapuram
<Satyanarayana.Narlapuram@microsoft.com> wrote:

I simplified the patch and for now just allowed one server. Please find the attached patches, and the commit message.

This patch --

-- doesn't include nearly sufficient documentation updates. For
example, the new message type is not documented in the list of message
types. The documentation of which messages are legal in which
contexts is not updated to mention this new message. The new
ConnStatusType is not documented (and is it really needed?). The
documentation for the new pg_hba.conf parameter does not explain how
to specify the alternate server to which you wish to connect.

-- includes no tests.

-- does include irrelevant whitespace differences.

-- doesn't include any provision for clients to fall back to 3.0 if
3.1 is not supported.

-- doesn't seem to have proper provisions for the server to handle
older clients. The code looks like just skips over hba.conf redirect
lines if the client is older, which seems like not what we want. The
proposed commit message claims we just go ahead and send redirects to
older clients that aren't expecting them, which is pretty much missing
the entire point of having minor protocol versions. I think the right
way to handle this case is to throw FATAL with the error text
suggesting the host/port to which the user should try connecting.

-- probably needs defenses against infinite redirect loops. Most
likely we should see how this is normally handled by HTTP clients and
do something similar here.

-- probably needs some way for clients to express unwillingness to
follow redirects. Possibly that could be combined with the previous
item by having a new connection parameter indicating the number of
redirects the client is willing to follow, with the default being,
say, 10 (browsers apparently have a limit of 10 or 20 for HTTP, but 20
seems overly generous for a database connection) and 0 disabling.

-- might need some defense against the redirected-to server getting
the same password as was sent to the original server. Is that a
security risk? Does HTTP have a rule about this?

-- might need some way for clients to discover whether they got
redirected and, if so, the server to which they were redirected. For
example, if I connect with psql, get redirected, and then type
\conninfo, do I get the information on the server to which I think I
connected, or the server to which I got redirected? Maybe we should
display both? If the connection gets retried, do we retry the
original server or the server to which we were redirected? I'd argue
for the former, but maybe other people think differently.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#8)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

Robert Haas <robertmhaas@gmail.com> writes:

-- might need some defense against the redirected-to server getting
the same password as was sent to the original server. Is that a
security risk? Does HTTP have a rule about this?

Without having read any of the previous discussion ... I'd say that if the
redirect info is placed in pg_hba.conf then I would expect a redirect to
happen before any authentication exchange, so that this is not an issue.
Perhaps it would be a good security measure for clients to refuse a
redirect once they've sent any auth-related messages.

But ... pg_hba.conf? Really? Surely that is a completely random and
inappropriate place to control redirection?

regards, tom lane

#10Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#9)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

On Tue, Feb 13, 2018 at 5:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

-- might need some defense against the redirected-to server getting
the same password as was sent to the original server. Is that a
security risk? Does HTTP have a rule about this?

Without having read any of the previous discussion ... I'd say that if the
redirect info is placed in pg_hba.conf then I would expect a redirect to
happen before any authentication exchange, so that this is not an issue.
Perhaps it would be a good security measure for clients to refuse a
redirect once they've sent any auth-related messages.

But ... pg_hba.conf? Really? Surely that is a completely random and
inappropriate place to control redirection?

Where would you suggest?

My thought was that if, for example, you migrated one database off of
a multiple database cluster to a new location, you'd want to redirect
anyone trying to connect to that database to the new server, so you
need to put the redirection facility someplace where we can make
decisions about whether or not to redirect based on rules involving
database names. The other things we expose in pg_hba.conf seem like
they could potentially be useful, too, although maybe less so. For
instance, if you've got several standbys (or several masters connected
via some MMR solution) you could redirect connections which come from
the "wrong" IP block to the server to which they are local. I think
of pg_hba.conf as a place where we decide what to do with connections,
and redirecting them seems like one thing we might decide to do.

I hadn't really thought deeply about whether redirection should happen
before or after authentication. For the most part, before seems
better, because it seems a bit silly to force people to authenticate
just so that you can tell them to go someplace else. Also, that would
lead to double authentication, which might for example result in
multiple password prompts, which users might either dislike or find
confusing. The only contrary argument that comes to mind is that
someone could argue that there's a security leakage --- if someone has
a redirect rule that only engages for a particular user or database
name, then you can perhaps guess that the user or database name exists
on the target system, or that in general it's one that they care
about. However, reject rules already have the same exposure.
Similarly, you might also be able to infer something based on the type
of authentication request that you get from the server. So I don't
see this argument as compelling.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#11Satyanarayana Narlapuram
Satyanarayana.Narlapuram@microsoft.com
In reply to: Robert Haas (#10)
1 attachment(s)
RE: [HACKERS] Client Connection redirection support for PostgreSQL

Please see the attached patch with the comments.

Changes in the patch:
A client-side PGREDIRECTLIMIT parameter has been introduced to control the maximum number of retries.
BE_v3.1 sends a ProtocolNegotiation message. FE_v3.1 downgrades to v3.0 upon receipt of this message.
FE falls back to v3.0 if 3.1 is not supported by the server.

I hadn't really thought deeply about whether redirection should happen before or after authentication. For the most part, before seems better, because it seems a bit silly to force people to authenticate just so that you can tell them to go someplace else. Also, that would lead to double authentication, which might for example result in multiple password prompts, which users might either dislike or find confusing.

Yes, redirection before authentication would avoid multiple password prompts.

Thanks,
Satya

Attachments:

redirection_v2.patchapplication/octet-stream; name=redirection_v2.patchDownload
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index 53832d0..e0ae721 100644
--- a/doc/src/sgml/client-auth.sgml
+++ b/doc/src/sgml/client-auth.sgml
@@ -495,6 +495,18 @@ hostnossl  <replaceable>database</replaceable>  <replaceable>user</replaceable>
        </varlistentry>
 
        <varlistentry>
+        <term><literal>redirect</literal></term>
+        <listitem>
+         <para>
+          Redirects the connection to the alternative server specified if it
+          matches the requested database user name and IP address. This is
+          only available for Protocol Version 3.1 and beyond.
+          See <xref linkend="auth-redirect"/> for details.
+         </para>
+        </listitem>
+       </varlistentry>
+
+       <varlistentry>
         <term><literal>ldap</literal></term>
         <listitem>
          <para>
@@ -624,7 +636,7 @@ hostnossl  <replaceable>database</replaceable>  <replaceable>user</replaceable>
    non-null <structfield>error</structfield> fields indicate problems in the
    corresponding lines of the file.
   </para>
- 
+
   <tip>
    <para>
     To connect to a particular database, a user must not only pass the
@@ -1434,6 +1446,29 @@ omicron         bryanh                  guest1
 
   </sect2>
 
+  <sect2 id="auth-redirect">
+   <title>Redirect Authentication</title>
+
+   <indexterm>
+    <primary>redirect</primary>
+   </indexterm>
+
+   <para>
+    This authentication mechanism works by specifying an alternative server to redirect
+    client connections to. The alternative server endpoint is specified by a hostname and port, separated by a single comma.
+   </para>
+
+   <para>
+    An example entry with <productname>redirect</productname> would look like the following:
+    host    all             all             127.0.0.1/32            redirect  <hostname>,<port>
+   </para>
+
+   <para>
+   Support for redirection is only available from wire protocol v3.1 upwards.
+   </para>
+
+  </sect2>
+
   <sect2 id="auth-ldap">
    <title>LDAP Authentication</title>
 
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 3cec9e0..5b58270 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -4747,6 +4747,79 @@ GSSResponse (F)
 
 <varlistentry>
 <term>
+RedirectClient (B)
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('M')
+</term>
+<listitem>
+<para>
+                Redirects client connection to alternative server.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Length of message contents in bytes, including self.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        String
+</term>
+<listitem>
+<para>
+                Parameter name "server" to identify the redirect target host.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        String
+</term>
+<listitem>
+<para>
+                Specifies the target host name.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        String
+</term>
+<listitem>
+<para>
+                Parameter name "port" to identify the redirect target port.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        String
+</term>
+<listitem>
+<para>
+                Specifies the target host's port.
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
 NegotiateProtocolVersion (B)
 </term>
 <listitem>
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 3014b17..a700a9b 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -206,6 +206,12 @@ static int pg_SSPI_make_upn(char *accountname,
 static int	CheckRADIUSAuth(Port *port);
 static int	PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
 
+/*----------------------------------------------------------------
+ * Connection Redirection
+ *----------------------------------------------------------------
+ */
+static int	SendAlternativeServerName(Port *port, char **logdetail);
+
 
 /*
  * Maximum accepted size of GSS and SSPI authentication tokens.
@@ -598,6 +604,9 @@ ClientAuthentication(Port *port)
 		case uaTrust:
 			status = STATUS_OK;
 			break;
+		case uaRedirect:
+			status = SendAlternativeServerName(port, &logdetail);
+			break;
 	}
 
 	if (ClientAuthentication_hook)
@@ -609,6 +618,28 @@ ClientAuthentication(Port *port)
 		auth_failed(port, status, logdetail);
 }
 
+/*
+ * Send alternative server information packet to the frontend.
+ */
+static int
+SendAlternativeServerName(Port *port, char **logdetail)
+{
+	StringInfoData buf;
+
+	CHECK_FOR_INTERRUPTS();
+
+	pq_beginmessage(&buf, 'M');
+	pq_sendstring(&buf, "server");
+	pq_sendstring(&buf, port->hba->alternativeservername);
+	pq_sendstring(&buf, "port");
+	pq_sendstring(&buf, port->hba->alternativeserverport);
+
+	pq_endmessage(&buf);
+	pq_flush();
+	proc_exit(0);
+
+	return STATUS_OK;
+}
 
 /*
  * Send an authentication request packet to the frontend.
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index acf625e..f3f7d6f 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -134,7 +134,8 @@ static const char *const UserAuthName[] =
 	"ldap",
 	"cert",
 	"radius",
-	"peer"
+	"peer",
+	"redirect"
 };
 
 
@@ -1358,6 +1359,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
 #endif
 	else if (strcmp(token->string, "radius") == 0)
 		parsedline->auth_method = uaRADIUS;
+	else if (strcmp(token->string, "redirect") == 0)
+		parsedline->auth_method = uaRedirect;
 	else
 	{
 		ereport(elevel,
@@ -1384,6 +1387,49 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
 		return NULL;
 	}
 
+	if (parsedline->auth_method == uaRedirect)
+	{
+		/* Get the alternative server name and port */
+		field = lnext(field);
+		if (!field)
+		{
+			ereport(elevel,
+					(errcode(ERRCODE_CONFIG_FILE_ERROR),
+					 errmsg("end-of-line before alternative server name"),
+					 errcontext("line %d of configuration file \"%s\"",
+								line_num, HbaFileName)));
+			*err_msg = "end-of-line before alternative server name";
+			return NULL;
+		}
+		tokens = lfirst(field);
+		if (tokens->length > 2)
+		{
+			ereport(elevel,
+					(errcode(ERRCODE_CONFIG_FILE_ERROR),
+					 errmsg("multiple values specified for alternative server"),
+					 errhint("Specify exactly one alternative server per line."),
+					 errcontext("line %d of configuration file \"%s\"",
+								line_num, HbaFileName)));
+			*err_msg = "multiple values specified for alternative server";
+			return NULL;
+		}
+
+		tokencell = list_head(tokens);
+		token = lfirst(tokencell);
+		parsedline->alternativeservername = pstrdup(token->string);
+
+		if (tokens->length == 2)
+		{
+			tokencell = lnext(tokencell);
+			token = lfirst(tokencell);
+			parsedline->alternativeserverport = pstrdup(token->string);
+		}
+		else
+		{
+			pg_itoa(DEF_PGPORT, parsedline->alternativeserverport);
+		}
+	}
+
 	/*
 	 * XXX: When using ident on local connections, change it to peer, for
 	 * backwards compatibility.
@@ -2100,6 +2146,21 @@ check_hba(hbaPort *port)
 		if (!check_role(port->user_name, roleid, hba->roles))
 			continue;
 
+		/*
+		 * Check the protocol version to see if the client supports
+		 * redirection
+		 */
+		if (hba->auth_method == uaRedirect &&
+			PG_PROTOCOL_MAJOR(port->proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST) ||
+			(PG_PROTOCOL_MAJOR(port->proto) == PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST) &&
+			 PG_PROTOCOL_MINOR(port->proto) < PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST)))
+			ereport(FATAL,
+					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+					 errmsg("Redirection is only supported by protocol version 3.1 and above. \
+							Try connecting to %s:%s directly",
+							hba->alternativeservername,
+							hba->alternativeserverport)));
+
 		/* Found a record that matched! */
 		port->hba = hba;
 		return;
diff --git a/src/backend/libpq/pg_hba.conf.sample b/src/backend/libpq/pg_hba.conf.sample
index c853e36..3af0972 100644
--- a/src/backend/libpq/pg_hba.conf.sample
+++ b/src/backend/libpq/pg_hba.conf.sample
@@ -43,7 +43,7 @@
 # directly connected to.
 #
 # METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
-# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert".
+# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius", "cert" or "redirect".
 # Note that "password" sends passwords in clear text; "md5" or
 # "scram-sha-256" are preferred since they send encrypted passwords.
 #
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 5f68f4c..1e4f2bd 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -38,8 +38,9 @@ typedef enum UserAuth
 	uaLDAP,
 	uaCert,
 	uaRADIUS,
-	uaPeer
-#define USER_AUTH_LAST uaPeer	/* Must be last value of this enum */
+	uaPeer,
+	uaRedirect
+#define USER_AUTH_LAST uaRedirect	/* Must be last value of this enum */
 } UserAuth;
 
 typedef enum IPCompareMethod
@@ -99,6 +100,8 @@ typedef struct HbaLine
 	char	   *radiusidentifiers_s;
 	List	   *radiusports;
 	char	   *radiusports_s;
+	char	   *alternativeservername;
+	char	   *alternativeserverport;
 } HbaLine;
 
 typedef struct IdentLine
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 77eebb0..26fa934 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
 #else
 #define DefaultSSLMode	"disable"
 #endif
+#define DefaultRedirectionLimit "2"
 
 /* ----------
  * Definition of the conninfo parameters and their fallback resources.
@@ -266,7 +267,7 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 
 	{"scram_channel_binding", NULL, DefaultSCRAMChannelBinding, NULL,
 		"SCRAM-Channel-Binding", "D",
-		21,	/* sizeof("tls-server-end-point") == 21 */
+		21,						/* sizeof("tls-server-end-point") == 21 */
 	offsetof(struct pg_conn, scram_channel_binding)},
 
 	/*
@@ -330,6 +331,11 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	{"redirect_limit", "PGREDIRECTLIMIT",
+		DefaultRedirectionLimit, NULL,
+		"Redirection-Count", "", 10,	/* strlen(INT32_MAX) == 10 */
+	offsetof(struct pg_conn, redirect_limit)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1805,11 +1811,11 @@ connectDBStart(PGconn *conn)
 #endif
 
 	/*
-	 * Set up to try to connect, with protocol 3.0 as the first attempt.
+	 * Set up to try to connect, with protocol 3.1 as the first attempt.
 	 */
 	conn->whichhost = 0;
 	conn->addr_cur = conn->connhost[0].addrlist;
-	conn->pversion = PG_PROTOCOL(3, 0);
+	conn->pversion = PG_PROTOCOL(3, 1);
 	conn->send_appname = true;
 	conn->status = CONNECTION_NEEDED;
 
@@ -2007,6 +2013,14 @@ PQconnectPoll(PGconn *conn)
 	int			optval;
 	PQExpBufferData savedMessage;
 
+	/* Variable declarations for redirection. */
+	int			originalMsgLen; /* Length in bytes of message sans msg type */
+	int			runningMsgLen;	/* Length in bytes of message sans metadata */
+	int			availableMsgLen;
+	char	   *altServer = NULL;
+	char	   *altPort = NULL;
+	bool		redirectionError = false;	/* Flag used to mark exceptions */
+
 	if (conn == NULL)
 		return PGRES_POLLING_FAILED;
 
@@ -2024,6 +2038,7 @@ PQconnectPoll(PGconn *conn)
 
 			/* These are reading states */
 		case CONNECTION_AWAITING_RESPONSE:
+		case CONNECTION_REDIRECTION:
 		case CONNECTION_AUTH_OK:
 			{
 				/* Load waiting data */
@@ -2655,6 +2670,25 @@ keep_going:						/* We will come back to here until there is
 					return PGRES_POLLING_READING;
 				}
 
+				if (beresp == 'M')
+				{
+					conn->status = CONNECTION_REDIRECTION;
+					goto keep_going;
+				}
+
+				/*
+				 * If server sends protocol negotiation message, default to
+				 * 3.0 protocol.
+				 */
+				if (beresp == 'v')
+				{
+					conn->pversion = PG_PROTOCOL(3, 0);
+					/* Must drop the old connection */
+					pqDropConnection(conn, true);
+					conn->status = CONNECTION_NEEDED;
+					goto keep_going;
+				}
+
 				/*
 				 * Validate message type: we expect only an authentication
 				 * request or an error here.  Anything else probably means
@@ -2722,19 +2756,28 @@ keep_going:						/* We will come back to here until there is
 					appendPQExpBufferChar(&conn->errorMessage, '\n');
 
 					/*
-					 * If we tried to open the connection in 3.0 protocol,
-					 * fall back to 2.0 protocol.
+					 * If we tried to open the connection in 3.1 protocol,
+					 * fall back to 3.0 protocol. If that fails as well, fall
+					 * back to 2.0 protocol.
 					 */
-					if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
+					if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3
+						&& PG_PROTOCOL_MINOR(conn->pversion) >= 1)
+					{
+						conn->pversion = PG_PROTOCOL(3, 0);
+					}
+					else if (PG_PROTOCOL_MAJOR(conn->pversion) >= 3)
 					{
 						conn->pversion = PG_PROTOCOL(2, 0);
-						/* Must drop the old connection */
-						pqDropConnection(conn, true);
-						conn->status = CONNECTION_NEEDED;
-						goto keep_going;
+					}
+					else
+					{
+						goto error_return;
 					}
 
-					goto error_return;
+					/* Must drop the old connection */
+					pqDropConnection(conn, true);
+					conn->status = CONNECTION_NEEDED;
+					goto keep_going;
 				}
 
 				/*
@@ -3097,6 +3140,146 @@ keep_going:						/* We will come back to here until there is
 				conn->status = CONNECTION_OK;
 				return PGRES_POLLING_OK;
 			}
+
+		case CONNECTION_REDIRECTION:
+			{
+				/*
+				 * Check if the number of redirect attempts exceeds the limit.
+				 */
+				if (++conn->nRedirection > atoi(conn->redirect_limit))
+				{
+					appendPQExpBuffer(&conn->errorMessage,
+									  libpq_gettext("Exceeded the maximum number of redirection attempts."));
+					goto error_return;
+				}
+
+				/* Mark 'M' consumed: a single byte for message type. */
+				conn->inCursor = conn->inStart + 1;
+
+				/* Obtain message length from packet. */
+				if (pqGetInt(&originalMsgLen, sizeof(int32), conn))
+					return PGRES_POLLING_READING;
+
+				/* Obtain the number of bytes in payload. */
+				runningMsgLen = originalMsgLen - sizeof(int32);
+
+				/*
+				 * Enlarge buffer if payload's size is greater than what is
+				 * available.
+				 */
+				availableMsgLen = conn->inEnd - conn->inCursor;
+				if (availableMsgLen < runningMsgLen)
+				{
+					if (pqCheckInBufferSpace(conn->inCursor + (size_t) runningMsgLen, conn))
+						return PGRES_POLLING_READING;
+				}
+
+				PQExpBuffer buf = createPQExpBuffer();
+
+				while (pqGets(buf, conn) != EOF)
+				{
+					if (!strcmp(buf->data, "server"))
+					{
+						if (pqGets(buf, conn) == EOF)
+						{
+							appendPQExpBuffer(&conn->errorMessage,
+											  libpq_gettext("failed to obtain server value from redirection packet"));
+
+							redirectionError = true;
+							break;
+						}
+						altServer = strdup(buf->data);
+					}
+					else if (!strcmp(buf->data, "port"))
+					{
+						if (pqGets(buf, conn) == EOF)
+						{
+							appendPQExpBuffer(&conn->errorMessage,
+											  libpq_gettext("failed to obtain port value from redirection packet"));
+
+							redirectionError = true;
+							break;
+						}
+						altPort = strdup(buf->data);
+					}
+					else
+					{
+						appendPQExpBuffer(&conn->errorMessage,
+										  libpq_gettext("unknown key in redirection server packet"));
+						redirectionError = true;
+					}
+
+					if (redirectionError)
+					{
+						/* Free buffer to prevent memory leak on error. */
+						destroyPQExpBuffer(buf);
+
+						/* Free strdup'd variables. */
+						if (altServer)
+							free(altServer);
+
+						if (altPort)
+							free(altPort);
+
+						goto error_return;
+					}
+
+					/*
+					 * Buffer length does not account for null-terminated
+					 * strings.
+					 */
+					runningMsgLen -= buf->len + 1;
+				}
+
+				/* Free buffer used for reading string params in packet. */
+				destroyPQExpBuffer(buf);
+
+				/* Check for extraneous data in packet. */
+				if (conn->inCursor != conn->inStart + 1 + originalMsgLen)
+				{
+					appendPQExpBuffer(&conn->errorMessage,
+									  libpq_gettext("Extraneous data in redirection packet from server\n"));
+
+					/* Free strdup'd variables. */
+					if (altServer)
+						free(altServer);
+
+					if (altPort)
+						free(altPort);
+
+					goto error_return;
+				}
+
+				/* Mark incoming data consumed */
+				conn->inStart = conn->inCursor;
+
+				/* Drop existing connection. */
+				pqDropConnection(conn, true);
+
+				/* Set connection parameters. */
+				if (conn->pghost)
+				{
+					free(conn->pghost);
+					conn->pghost = altServer;
+				}
+
+				if (conn->pgport)
+				{
+					free(conn->pgport);
+					conn->pgport = altPort;
+				}
+
+				/* connectDBStart() sets appropriate connection status. */
+				if (!connectOptions2(conn) || !connectDBStart(conn))
+				{
+					conn->status = CONNECTION_BAD;
+					appendPQExpBuffer(&conn->errorMessage,
+									  libpq_gettext("Failed to establish connection to redirected database\n"));
+				}
+
+				goto keep_going;
+			}
+
 		case CONNECTION_CHECK_WRITABLE:
 			{
 				const char *displayed_host;
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index ed9c806..38616d2 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -65,8 +65,10 @@ typedef enum
 	CONNECTION_NEEDED,			/* Internal state: connect() needed */
 	CONNECTION_CHECK_WRITABLE,	/* Check if we could make a writable
 								 * connection. */
-	CONNECTION_CONSUME			/* Wait for any pending message and consume
+	CONNECTION_CONSUME,			/* Wait for any pending message and consume
 								 * them. */
+	CONNECTION_REDIRECTION		/* Redirecting the connection to the
+								 * alternative server specified in pg_hba.conf */
 } ConnStatusType;
 
 typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index eba23dc..d4edd40 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -349,7 +349,7 @@ struct pg_conn
 										 * retransmits */
 	char	   *keepalives_count;	/* maximum number of TCP keepalive
 									 * retransmits */
-	char	   *scram_channel_binding; /* SCRAM channel binding type */
+	char	   *scram_channel_binding;	/* SCRAM channel binding type */
 	char	   *sslmode;		/* SSL mode (require,prefer,allow,disable) */
 	char	   *sslcompression; /* SSL compression (0 or 1) */
 	char	   *sslkey;			/* client key filename */
@@ -495,6 +495,10 @@ struct pg_conn
 
 	/* Buffer for receiving various parts of messages */
 	PQExpBufferData workBuffer; /* expansible string */
+
+	char	   *redirect_limit; /* Specifies the maximum number of times to
+								 * attempt redirection. */
+	int			nRedirection;	/* Number of redirects attempted so far */
 };
 
 /* PGcancel stores all data necessary to cancel a connection. A copy of this
@@ -742,8 +746,8 @@ extern char *pgtls_get_peer_certificate_hash(PGconn *conn, size_t *len);
  *
  */
 extern int pgtls_verify_peer_name_matches_certificate_guts(PGconn *conn,
-														   int *names_examined,
-														   char **first_name);
+												int *names_examined,
+												char **first_name);
 
 /* === miscellaneous macros === */
 
diff --git a/src/test/authentication/t/003_redirection.pl b/src/test/authentication/t/003_redirection.pl
new file mode 100644
index 0000000..db0b78c
--- /dev/null
+++ b/src/test/authentication/t/003_redirection.pl
@@ -0,0 +1,92 @@
+# Test redirection authentication method.
+#
+# This test cannot run on Windows as Postgres cannot be set up with Unix
+# sockets and needs to go through SSPI.
+
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More;
+if ($windows_os)
+{
+	plan skip_all => "authentication tests cannot run on Windows";
+}
+else
+{
+	plan tests => 3;
+}
+
+# Delete pg_hba.conf from the given node, add a new entry to it
+# and then execute a reload to refresh it.
+sub reset_pg_hba
+{
+	my $node       = shift;
+	my $hba_method = shift;
+	my $redirect_endpoint = shift;
+
+	unlink($node->data_dir . '/pg_hba.conf');
+	$node->append_conf('pg_hba.conf', "local all all $hba_method $redirect_endpoint");
+	$node->reload;
+}
+
+# Test access for a single role, useful to wrap all tests into one.
+sub test_login
+{
+	my $node          = shift;
+	my $role          = shift;
+	my $password      = shift;
+	my $expected_res  = shift;
+	my $status_string = 'failed';
+
+	$status_string = 'success' if ($expected_res eq 0);
+
+	$ENV{"PGPASSWORD"} = $password;
+	my $res = $node->psql('postgres', undef, extra_params => [ '-U', $role ]);
+	is($res, $expected_res,
+		"authentication $status_string for role $role with password $password"
+	);
+}
+
+# Initialize node1.
+my $node1 = get_new_node('master');
+$node1->init;
+$node1->start;
+$node1->safe_psql(
+	'postgres',
+	"CREATE ROLE pguser1 LOGIN PASSWORD 'postgres';
+");
+
+# Initialize node2, the redirect target.
+my $node2 = get_new_node('redirect');
+$node2->init;
+$node2->start;
+$node2->safe_psql(
+	'postgres',
+	"CREATE ROLE pguser2 LOGIN PASSWORD 'postgres';
+");
+
+# Host is identical as both nodes reside on the same machine
+my $host = $node1->host;
+
+my $node1_port = $node1->port;
+# 1. Test a redirected connection from node1 to itself.
+# Add the redirect authentication method to the node1's pg_hba.conf to set up redirection to itself.
+reset_pg_hba($node1, 'redirect', "$host,$node1_port");
+# A redirect from a node to itself should fail after PGREDIRECTLIMIT (default is 5) retries.
+test_login($node1, 'pguser1', "postgres",   2);
+
+
+my $node2_port = $node2->port;
+# 2. Test a redirected connection from node1 to node2 with correct creds.
+# Add the redirect authentication method to the node1's pg_hba.conf to set up redirection to node2.
+reset_pg_hba($node1, 'redirect', "$host,$node2_port");
+# A redirect from a node to another should succeed, given the correct creds are used.
+test_login($node1, 'pguser2', "postgres",   0);
+
+
+# 3. Test a redirected connection from node1 to node2 with wrong creds.
+# Add the redirect authentication method to the node1's pg_hba.conf to set up redirection to node2.
+reset_pg_hba($node1, 'redirect', "$host,$node2_port");
+# A redirect from a node to another should fail if the wrong creds are used.
+test_login($node1, 'pguser', "postgres",   2);
#12Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Satyanarayana Narlapuram (#11)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

On 05/03/18 22:18, Satyanarayana Narlapuram wrote:

Please see the attached patch with the comments.

Changes in the patch:
A client-side PGREDIRECTLIMIT parameter has been introduced to control the maximum number of retries.
BE_v3.1 sends a ProtocolNegotiation message. FE_v3.1 downgrades to v3.0 upon receipt of this message.
FE falls back to v3.0 if 3.1 is not supported by the server.

I hadn't really thought deeply about whether redirection should happen before or after authentication. For the most part, before seems better, because it seems a bit silly to force people to authenticate just so that you can tell them to go someplace else. Also, that would lead to double authentication, which might for example result in multiple password prompts, which users might either dislike or find confusing.

Yes, redirection before authentication would avoid multiple password prompts.

I think we should have this feature. I can see a redirect being useful
in some similar cases like HTTP redirects are useful, but a database
server is not a web server. There are no redirects in IMAP or most other
protocols, either.

This would also require modifying every client library to honor the
redirect.

How would the redirect behave with TLS certificate verification? If you
are redirected from "foo-server" to "bar-server", but the original
connection string was "host=foo-server sslmode=verify-full", would the
connection be allowed?

FWIW, if we were to do this, I think pg_hba.conf would be a fine place
for this. That's where you currently have configuration for what happens
when a client with certain host/username/database tries to connect. In
addition to "accept" or "reject", it seems logical to add "redirect" as
an outcome, instead of e.g. adding a whole new configuration file fore this.

But overall, IMHO we should mark this patch "rejected".

- Heikki

#13Andres Freund
andres@anarazel.de
In reply to: Heikki Linnakangas (#12)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

On 2018-07-13 23:00:04 +0300, Heikki Linnakangas wrote:

On 05/03/18 22:18, Satyanarayana Narlapuram wrote:

Please see the attached patch with the comments.

Changes in the patch:
A client-side PGREDIRECTLIMIT parameter has been introduced to control the maximum number of retries.
BE_v3.1 sends a ProtocolNegotiation message. FE_v3.1 downgrades to v3.0 upon receipt of this message.
FE falls back to v3.0 if 3.1 is not supported by the server.

I hadn't really thought deeply about whether redirection should happen before or after authentication. For the most part, before seems better, because it seems a bit silly to force people to authenticate just so that you can tell them to go someplace else. Also, that would lead to double authentication, which might for example result in multiple password prompts, which users might either dislike or find confusing.

Yes, redirection before authentication would avoid multiple password prompts.

FWIW, I think it's quite dangerous to do the redirect before
authentication, and more importantly, certificate validation / channel
binding.

FWIW, if we were to do this, I think pg_hba.conf would be a fine place for
this. That's where you currently have configuration for what happens when a
client with certain host/username/database tries to connect. In addition to
"accept" or "reject", it seems logical to add "redirect" as an outcome,
instead of e.g. adding a whole new configuration file fore this.

I'd personally argue that it'd also make sense to have this as actual
database level option.

One thing where I can see a feature like this being quite helpful is
planned failovers, reducing the time to reconnect (for existing
connections) and rediscover (for new connections, which need to
write). But that'd require that the redirect needs to be able to be sent
in an established connection too.

Greetings,

Andres Freund

#14Dave Cramer
pg@fastcrypt.com
In reply to: Andres Freund (#13)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

One thing where I can see a feature like this being quite helpful is
planned failovers, reducing the time to reconnect (for existing
connections) and rediscover (for new connections, which need to
write). But that'd require that the redirect needs to be able to be sent
in an established connection too.

Somewhat related to this feature is the notion of giving a hint as to
whether a connection is read only.

Currently we can create a read only transaction which at this point pgpool
through some machinations which are less than favourable IMO
can connect to a secondary. This even works with the JDBC driver which has
setReadOnly facility on connections.

However it would be far better to have a startup parameter which indicated
that we wanted to connect to a read only database. At that point
pools could redirect to a secondary. Given the proliferation of cloud based
implementations I can see this being a useful feature.

Regards,

Dave Cramer

#15Michael Paquier
michael@paquier.xyz
In reply to: Dave Cramer (#14)
Re: [HACKERS] Client Connection redirection support for PostgreSQL

On Fri, Jul 13, 2018 at 04:20:15PM -0400, Dave Cramer wrote:

However it would be far better to have a startup parameter which indicated
that we wanted to connect to a read only database. At that point
pools could redirect to a secondary. Given the proliferation of cloud based
implementations I can see this being a useful feature.

The thread has died here... Without any input from the author for many
months. So I am marking it as returned with feedback.
--
Michael