[PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

Started by Phil Sorberalmost 13 years ago25 messages
#1Phil Sorber
phil@omniti.com

This patch came up from discussion about pg_isready.

PQconninfoParseParams is similar to PQconninfoParse but takes two
arrays like PQconnectdbParams. It essentially exposes
conninfo_array_parse().

PQconninfodefaultsMerge essentially exposes conninfo_add_defaults().
It allows you to pass a PQconninfoOption struct and it adds defaults
for all NULL values.

There are no docs yet. I assumed I would let bikeshedding ensue, and
also debate on whether we even want these first.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Magnus Hagander
magnus@hagander.net
In reply to: Phil Sorber (#1)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Feb 3, 2013 4:16 AM, "Phil Sorber" <phil@omniti.com> wrote:

This patch came up from discussion about pg_isready.

PQconninfoParseParams is similar to PQconninfoParse but takes two
arrays like PQconnectdbParams. It essentially exposes
conninfo_array_parse().

PQconninfodefaultsMerge essentially exposes conninfo_add_defaults().
It allows you to pass a PQconninfoOption struct and it adds defaults
for all NULL values.

There are no docs yet. I assumed I would let bikeshedding ensue, and
also debate on whether we even want these first.

I think you forgot to attach the patch.

/Magnus

#3Phil Sorber
phil@omniti.com
In reply to: Magnus Hagander (#2)
1 attachment(s)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Sun, Feb 3, 2013 at 1:37 AM, Magnus Hagander <magnus@hagander.net> wrote:

On Feb 3, 2013 4:16 AM, "Phil Sorber" <phil@omniti.com> wrote:

This patch came up from discussion about pg_isready.

PQconninfoParseParams is similar to PQconninfoParse but takes two
arrays like PQconnectdbParams. It essentially exposes
conninfo_array_parse().

PQconninfodefaultsMerge essentially exposes conninfo_add_defaults().
It allows you to pass a PQconninfoOption struct and it adds defaults
for all NULL values.

There are no docs yet. I assumed I would let bikeshedding ensue, and
also debate on whether we even want these first.

I think you forgot to attach the patch.

/Magnus

/me hangs head in shame :-(

Here is is...

Attachments:

libpq_params_parse_merge.diffapplication/octet-stream; name=libpq_params_parse_merge.diffDownload
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
new file mode 100644
index 93da50d..5566943
*** a/src/interfaces/libpq/exports.txt
--- b/src/interfaces/libpq/exports.txt
*************** lo_lseek64                162
*** 165,167 ****
--- 165,169 ----
  lo_tell64                 163
  lo_truncate64             164
  PQconninfo                165
+ PQconninfoParseParams     166
+ PQconninfodefaultsMerge   167
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
new file mode 100644
index eea9c6b..5546c38
*** a/src/interfaces/libpq/fe-connect.c
--- b/src/interfaces/libpq/fe-connect.c
*************** PQconninfoParse(const char *conninfo, ch
*** 4002,4007 ****
--- 4002,4085 ----
  }
  
  /*
+  *		PQconninfoParseParams
+  *
+  * Parse a string like PQconnectdbParams() would do and return the
+  * resulting connection options array.	NULL is returned on failure.
+  * The result contains only options specified directly in the array,
+  * not any possible default values.
+  *
+  * If errmsg isn't NULL, *errmsg is set to NULL on success, or a malloc'd
+  * string on failure (use PQfreemem to free it).  In out-of-memory conditions
+  * both *errmsg and the result could be NULL.
+  *
+  * NOTE: the returned array is dynamically allocated and should
+  * be freed when no longer needed via PQconninfoFree().
+  */
+ PQconninfoOption *PQconninfoParseParams(const char *const * keywords,
+ 				  const char *const * values, int expand_dbname,
+ 				  char **errmsg)
+ {
+ 	PQExpBufferData errorBuf;
+ 	PQconninfoOption *connOptions;
+ 
+ 	if (errmsg)
+ 		*errmsg = NULL;			/* default */
+ 	initPQExpBuffer(&errorBuf);
+ 	if (PQExpBufferDataBroken(errorBuf))
+ 		return NULL;			/* out of memory already :-( */
+ 	connOptions = conninfo_array_parse(keywords, values,
+ 									   &errorBuf,
+ 									   false, expand_dbname);
+ 
+ 	if (connOptions == NULL && errmsg)
+ 		*errmsg = errorBuf.data;
+ 	else
+ 		termPQExpBuffer(&errorBuf);
+ 	return connOptions;
+ }
+ 
+ /*
+  *		PQconninfodefaultsMerge
+  *
+  * Add the default values for any unspecified options to the connection
+  * options array.
+  *
+  * Defaults are obtained from a service file, environment variables, etc.
+  *
+  * Returns 1 if successful, otherwise 0; connOptions is freed and errmsg is
+  * filled in upon failure.  Note that failure to locate a default value is
+  * not an error condition here --- we just leave the option's value as NULL.
+  *
+  * If errmsg isn't NULL, *errmsg is set to NULL on success, or a malloc'd
+  * string on failure (use PQfreemem to free it).  In out-of-memory conditions
+  * both *errmsg and the result could be NULL.
+  */
+ int
+ PQconninfodefaultsMerge(PQconninfoOption *connOptions, char **errmsg)
+ {
+ 	PQExpBufferData errorBuf;
+ 
+ 	if (errmsg)
+ 		*errmsg = NULL;
+ 	initPQExpBuffer(&errorBuf);
+ 	if (PQExpBufferDataBroken(errorBuf))
+ 	{
+ 		PQconninfoFree(connOptions);
+ 		return 0;
+ 	}
+ 	if (!conninfo_add_defaults(connOptions, &errorBuf))
+ 	{
+ 		if (errmsg)
+ 			*errmsg = errorBuf.data;
+ 		PQconninfoFree(connOptions);
+ 		return 0;
+ 	}
+ 	termPQExpBuffer(&errorBuf);
+ 	return 1;
+ }
+ 
+ /*
   * Build a working copy of the constant PQconninfoOptions array.
   */
  static PQconninfoOption *
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
new file mode 100644
index e0f4bc7..9fa3c1c
*** a/src/interfaces/libpq/libpq-fe.h
--- b/src/interfaces/libpq/libpq-fe.h
*************** extern PQconninfoOption *PQconndefaults(
*** 267,276 ****
  /* parse connection options in same way as PQconnectdb */
  extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
  
  /* return the connection options used by a live connection */
  extern PQconninfoOption *PQconninfo(PGconn *conn);
  
! /* free the data structure returned by PQconndefaults() or PQconninfoParse() */
  extern void PQconninfoFree(PQconninfoOption *connOptions);
  
  /*
--- 267,284 ----
  /* parse connection options in same way as PQconnectdb */
  extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
  
+ /* parse connection options in same way as PQconnectdbParams */
+ extern PQconninfoOption *PQconninfoParseParams(const char *const * keywords,
+ 				  const char *const * values, int expand_dbname,
+ 				  char **errmsg);
+ 
+ /* Merge default connection options into an existing PQconninfoOption struct */
+ extern int PQconninfodefaultsMerge(PQconninfoOption *connOptions, char **errmsg);
+ 
  /* return the connection options used by a live connection */
  extern PQconninfoOption *PQconninfo(PGconn *conn);
  
! /* free the data structure returned by PQconndefaults(), PQconninfoParse(), or PQconninfoParseParams() */
  extern void PQconninfoFree(PQconninfoOption *connOptions);
  
  /*
#4Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Phil Sorber (#3)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Feb 3, 2013 4:16 AM, "Phil Sorber" <phil@omniti.com> wrote:

This patch came up from discussion about pg_isready.

PQconninfoParseParams is similar to PQconninfoParse but takes two
arrays like PQconnectdbParams. It essentially exposes
conninfo_array_parse().

PQconninfodefaultsMerge essentially exposes conninfo_add_defaults().
It allows you to pass a PQconninfoOption struct and it adds defaults
for all NULL values.

Uh, no existing code can use this new functionality? That seems
disappointing.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Phil Sorber
phil@omniti.com
In reply to: Alvaro Herrera (#4)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

On Feb 3, 2013 4:16 AM, "Phil Sorber" <phil@omniti.com> wrote:

This patch came up from discussion about pg_isready.

PQconninfoParseParams is similar to PQconninfoParse but takes two
arrays like PQconnectdbParams. It essentially exposes
conninfo_array_parse().

PQconninfodefaultsMerge essentially exposes conninfo_add_defaults().
It allows you to pass a PQconninfoOption struct and it adds defaults
for all NULL values.

Uh, no existing code can use this new functionality? That seems
disappointing.

I wrote this because I wanted to use it in pg_isready. I also wrote
something for pg_isready to get around not having this. I think adding
these two functions to libpq would be the better option, but wanted
something that could fix an existing issue without having to patch
libpq so late in the 9.3 development process. Actually, I think it was
you who suggested that approach.

So long answer short, there is existing code that can use this
functionality immediately.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Phil Sorber (#5)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

Uh, no existing code can use this new functionality? That seems
disappointing.

I wrote this because I wanted to use it in pg_isready. I also wrote
something for pg_isready to get around not having this. I think adding
these two functions to libpq would be the better option, but wanted
something that could fix an existing issue without having to patch
libpq so late in the 9.3 development process. Actually, I think it was
you who suggested that approach.

Yes, I realize that (and yes, I did). But is no code other than
pg_isready doing this? Not even the libpq URI test program?

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Phil Sorber
phil@omniti.com
In reply to: Alvaro Herrera (#6)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

Uh, no existing code can use this new functionality? That seems
disappointing.

I wrote this because I wanted to use it in pg_isready. I also wrote
something for pg_isready to get around not having this. I think adding
these two functions to libpq would be the better option, but wanted
something that could fix an existing issue without having to patch
libpq so late in the 9.3 development process. Actually, I think it was
you who suggested that approach.

Yes, I realize that (and yes, I did). But is no code other than
pg_isready doing this? Not even the libpq URI test program?

I think it probably would be able to benefit from this. Are you
suggesting I patch that too? I thought it was usually frowned upon to
touch random bits of working code like that. I'd be more than happy to
do it if it helps build the case for getting this added.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Phil Sorber (#7)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

Uh, no existing code can use this new functionality? That seems
disappointing.

I wrote this because I wanted to use it in pg_isready. I also wrote
something for pg_isready to get around not having this. I think adding
these two functions to libpq would be the better option, but wanted
something that could fix an existing issue without having to patch
libpq so late in the 9.3 development process. Actually, I think it was
you who suggested that approach.

Yes, I realize that (and yes, I did). But is no code other than
pg_isready doing this? Not even the libpq URI test program?

I think it probably would be able to benefit from this. Are you
suggesting I patch that too? I thought it was usually frowned upon to
touch random bits of working code like that. I'd be more than happy to
do it if it helps build the case for getting this added.

Well, this qualifies as refactoring, so yeah, it helps the case.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Heikki Linnakangas
hlinnakangas@vmware.com
In reply to: Alvaro Herrera (#8)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On 04.02.2013 17:32, Alvaro Herrera wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera<alvherre@2ndquadrant.com> wrote:

Uh, no existing code can use this new functionality? That seems
disappointing.

I wrote this because I wanted to use it in pg_isready. I also wrote
something for pg_isready to get around not having this. I think adding
these two functions to libpq would be the better option, but wanted
something that could fix an existing issue without having to patch
libpq so late in the 9.3 development process. Actually, I think it was
you who suggested that approach.

Yes, I realize that (and yes, I did). But is no code other than
pg_isready doing this? Not even the libpq URI test program?

I think it probably would be able to benefit from this. Are you
suggesting I patch that too? I thought it was usually frowned upon to
touch random bits of working code like that. I'd be more than happy to
do it if it helps build the case for getting this added.

Well, this qualifies as refactoring, so yeah, it helps the case.

I think this patch would simplift the patch to pass a connection string
to pg_basebackup and pg_receivexlog:
/messages/by-id/005501cdfa45$6b0eec80$412cc580$@kommi@huawei.com.
I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it either.

If you could come up with a unified patch that takes care of all of
those, that would be great.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Phil Sorber
phil@omniti.com
In reply to: Heikki Linnakangas (#9)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Mon, Feb 11, 2013 at 4:19 PM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:

On 04.02.2013 17:32, Alvaro Herrera wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro
Herrera<alvherre@2ndquadrant.com> wrote:

Uh, no existing code can use this new functionality? That seems
disappointing.

I wrote this because I wanted to use it in pg_isready. I also wrote
something for pg_isready to get around not having this. I think adding
these two functions to libpq would be the better option, but wanted
something that could fix an existing issue without having to patch
libpq so late in the 9.3 development process. Actually, I think it was
you who suggested that approach.

Yes, I realize that (and yes, I did). But is no code other than
pg_isready doing this? Not even the libpq URI test program?

I think it probably would be able to benefit from this. Are you
suggesting I patch that too? I thought it was usually frowned upon to
touch random bits of working code like that. I'd be more than happy to
do it if it helps build the case for getting this added.

Well, this qualifies as refactoring, so yeah, it helps the case.

I think this patch would simplift the patch to pass a connection string to
pg_basebackup and pg_receivexlog:
/messages/by-id/005501cdfa45$6b0eec80$412cc580$@kommi@huawei.com.
I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it either.

If you could come up with a unified patch that takes care of all of those,
that would be great.

I will take a look.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Amit kapila
amit.kapila@huawei.com
In reply to: Heikki Linnakangas (#9)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Tuesday, February 12, 2013 2:49 AM Heikki Linnakangas wrote:
On 04.02.2013 17:32, Alvaro Herrera wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera<alvherre@2ndquadrant.com> wrote:

I think this patch would simplift the patch to pass a connection string
to pg_basebackup and pg_receivexlog:
/messages/by-id/005501cdfa45$6b0eec80$412cc580$@kommi@huawei.com.
I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it either.

I have looked into both patches and my analysis is as below:

In pg_basebackup patch, we have connection string and list of keywords (individual options specified by user),
in the current available patch it has combined connection string and list of keywords as connection string
and called PQconnectdb() which takes connection string as input.

Now the patch of Phil Sober provides 2 new API's PQconninfoParseParams(), and PQconninfodefaultsMerge(),
using these API's I can think of below way for patch "pass a connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by user) and extract the keywords from
PQconninfoOption structure and call new API PQconninfoParseParams() which will return PQconninfoOption.
The PQconninfoOption structure returned in this step will contain all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure and call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of some other simpler way of using new API's?

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Phil Sorber
phil@omniti.com
In reply to: Amit kapila (#11)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Sun, Feb 17, 2013 at 1:35 AM, Amit kapila <amit.kapila@huawei.com> wrote:

On Tuesday, February 12, 2013 2:49 AM Heikki Linnakangas wrote:
On 04.02.2013 17:32, Alvaro Herrera wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro Herrera<alvherre@2ndquadrant.com> wrote:

I think this patch would simplift the patch to pass a connection string
to pg_basebackup and pg_receivexlog:
/messages/by-id/005501cdfa45$6b0eec80$412cc580$@kommi@huawei.com.
I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it either.

I have looked into both patches and my analysis is as below:

In pg_basebackup patch, we have connection string and list of keywords (individual options specified by user),
in the current available patch it has combined connection string and list of keywords as connection string
and called PQconnectdb() which takes connection string as input.

Now the patch of Phil Sober provides 2 new API's PQconninfoParseParams(), and PQconninfodefaultsMerge(),
using these API's I can think of below way for patch "pass a connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by user) and extract the keywords from
PQconninfoOption structure and call new API PQconninfoParseParams() which will return PQconninfoOption.
The PQconninfoOption structure returned in this step will contain all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure and call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of some other simpler way of using new API's?

With Regards,
Amit Kapila.

I think what would be nice is an additional connect function that took
PQconninfoOption as a parameter. Or at least something that can
convert from PQconninfoOption to a connection string or keyword/value
arrays.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Amit Kapila
amit.kapila@huawei.com
In reply to: Phil Sorber (#12)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote:

On Sun, Feb 17, 2013 at 1:35 AM, Amit kapila <amit.kapila@huawei.com>
wrote:

On Tuesday, February 12, 2013 2:49 AM Heikki Linnakangas wrote:
On 04.02.2013 17:32, Alvaro Herrera wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 10:16 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Phil Sorber wrote:

On Mon, Feb 4, 2013 at 9:13 AM, Alvaro

Herrera<alvherre@2ndquadrant.com> wrote:

I think this patch would simplift the patch to pass a connection

string

to pg_basebackup and pg_receivexlog:
http://www.postgresql.org/message-

id/005501cdfa45$6b0eec80$412cc580$@kommi@huawei.com.

I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it

either.

I have looked into both patches and my analysis is as below:

In pg_basebackup patch, we have connection string and list of

keywords (individual options specified by user),

in the current available patch it has combined connection string and

list of keywords as connection string

and called PQconnectdb() which takes connection string as input.

Now the patch of Phil Sober provides 2 new API's

PQconninfoParseParams(), and PQconninfodefaultsMerge(),

using these API's I can think of below way for patch "pass a

connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string

input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by

user) and extract the keywords from

PQconninfoOption structure and call new API

PQconninfoParseParams() which will return PQconninfoOption.

The PQconninfoOption structure returned in this step will contain

all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if

exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure and

call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of some

other simpler way of using new API's?

With Regards,
Amit Kapila.

I think what would be nice is an additional connect function that took
PQconninfoOption as a parameter. Or at least something that can
convert from PQconninfoOption to a connection string or keyword/value
arrays.

Yes, it would be better if we would like to use new API's, I think it can
save step-4 and some part in step-2.
I am not sure for this purpose would it be okay to introduce new Connect
API?

I also feel it will increase the scope of patch.

Heikki, would you be more specific that what in the patch you want to see
simplified.
Is the combining of keywords and connection string makes you feel the area
where patch can be improved.

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Heikki Linnakangas
hlinnakangas@vmware.com
In reply to: Amit Kapila (#13)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On 18.02.2013 06:07, Amit Kapila wrote:

On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote:

On Sun, Feb 17, 2013 at 1:35 AM, Amit kapila<amit.kapila@huawei.com>
wrote:

Now the patch of Phil Sober provides 2 new API's

PQconninfoParseParams(), and PQconninfodefaultsMerge(),

using these API's I can think of below way for patch "pass a

connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string

input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by

user) and extract the keywords from

PQconninfoOption structure and call new API

PQconninfoParseParams() which will return PQconninfoOption.

The PQconninfoOption structure returned in this step will contain

all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if

exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure and

call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of some

other simpler way of using new API's?

Yep, that's roughly what I had in mind. I don't think it's necessary to
merge defaults in step 3, but it needs to add the "replication=true" and
"dbname=replication" options.

I think what would be nice is an additional connect function that took
PQconninfoOption as a parameter. Or at least something that can
convert from PQconninfoOption to a connection string or keyword/value
arrays.

Yes, it would be better if we would like to use new API's, I think it can
save step-4 and some part in step-2.

pg_basebackup needs to add options to the array, so I don't think a new
connect function would help much. It's not a lot of code to iterate
through the PGconnInfoOption array and turn it back into keywords and
values arrays, so I'd just do that straight in the client code.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Amit Kapila
amit.kapila@huawei.com
In reply to: Heikki Linnakangas (#14)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Monday, February 18, 2013 1:41 PM Heikki Linnakangas wrote:

On 18.02.2013 06:07, Amit Kapila wrote:

On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote:

On Sun, Feb 17, 2013 at 1:35 AM, Amit kapila<amit.kapila@huawei.com>
wrote:

Now the patch of Phil Sober provides 2 new API's

PQconninfoParseParams(), and PQconninfodefaultsMerge(),

using these API's I can think of below way for patch "pass a

connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string

input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by

user) and extract the keywords from

PQconninfoOption structure and call new API

PQconninfoParseParams() which will return PQconninfoOption.

The PQconninfoOption structure returned in this step will

contain

all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if

exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure and

call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of

some

other simpler way of using new API's?

Yep, that's roughly what I had in mind. I don't think it's necessary to
merge defaults in step 3, but it needs to add the "replication=true"
and
"dbname=replication" options.

I think what would be nice is an additional connect function that

took

PQconninfoOption as a parameter. Or at least something that can
convert from PQconninfoOption to a connection string or

keyword/value

arrays.

Yes, it would be better if we would like to use new API's, I think it

can

save step-4 and some part in step-2.

pg_basebackup needs to add options to the array, so I don't think a new
connect function would help much. It's not a lot of code to iterate
through the PGconnInfoOption array and turn it back into keywords and
values arrays, so I'd just do that straight in the client code.

Okay, got the point.

Phil, I will try to finish the combined patch. Is it possible for you to
complete
the documentation for the new API's.

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#16Amit Kapila
amit.kapila@huawei.com
In reply to: Heikki Linnakangas (#14)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Monday, February 18, 2013 1:41 PM Heikki Linnakangas wrote:

On 18.02.2013 06:07, Amit Kapila wrote:

On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote:

On Sun, Feb 17, 2013 at 1:35 AM, Amit kapila<amit.kapila@huawei.com>
wrote:

Now the patch of Phil Sober provides 2 new API's

PQconninfoParseParams(), and PQconninfodefaultsMerge(),

using these API's I can think of below way for patch "pass a

connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string

input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by

user) and extract the keywords from

PQconninfoOption structure and call new API

PQconninfoParseParams() which will return PQconninfoOption.

The PQconninfoOption structure returned in this step will

contain

all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if

exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure and

call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of

some

other simpler way of using new API's?

Yep, that's roughly what I had in mind. I don't think it's necessary to
merge defaults in step 3, but it needs to add the "replication=true"
and
"dbname=replication" options.

I could see the advantage of calling PQconninfoParseParams() in step-2 is
that
it will remove the duplicate values by overriding the values for conflicting
keywords.
This is done in function conninfo_array_parse() which is called from
PQconninfoParseParams().
Am I right or there is any other advantage of calling
PQconninfoParseParams()?

If there is no other advantage then this is done in PQconnectdbParams()
also, so can't we avoid calling PQconninfoParseParams()?

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#17Amit Kapila
amit.kapila@huawei.com
In reply to: Amit Kapila (#16)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-
owner@postgresql.org] On Behalf Of Amit Kapila
Sent: Monday, February 18, 2013 6:38 PM
To: 'Heikki Linnakangas'
Cc: 'Phil Sorber'; 'Alvaro Herrera'; 'Magnus Hagander'; 'PostgreSQL-
development'
Subject: Re: [HACKERS] [PATCH] Add PQconninfoParseParams and
PQconninfodefaultsMerge to libpq

On Monday, February 18, 2013 1:41 PM Heikki Linnakangas wrote:

On 18.02.2013 06:07, Amit Kapila wrote:

On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote:

On Sun, Feb 17, 2013 at 1:35 AM, Amit

kapila<amit.kapila@huawei.com>

wrote:

Now the patch of Phil Sober provides 2 new API's

PQconninfoParseParams(), and PQconninfodefaultsMerge(),

using these API's I can think of below way for patch "pass a

connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection string

input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified by

user) and extract the keywords from

PQconninfoOption structure and call new API

PQconninfoParseParams() which will return PQconninfoOption.

The PQconninfoOption structure returned in this step will

contain

all keywords

3. Call PQconninfodefaultsMerge() to merge any default values if

exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure

and

call PQconnectdbParams.

Is this inline with what you have in mind or you have thought of

some

other simpler way of using new API's?

Yep, that's roughly what I had in mind. I don't think it's necessary

to

merge defaults in step 3, but it needs to add the "replication=true"
and
"dbname=replication" options.

I could see the advantage of calling PQconninfoParseParams() in step-2
is
that
it will remove the duplicate values by overriding the values for
conflicting
keywords.
This is done in function conninfo_array_parse() which is called from
PQconninfoParseParams().
Am I right or there is any other advantage of calling
PQconninfoParseParams()?

If there is no other advantage then this is done in PQconnectdbParams()
also, so can't we avoid calling PQconninfoParseParams()?

----

I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it
either.

I think not only pg_dumpall, but we need to add it to pg_dump.
As -C is already used option in pg_dump, I need to use something different.
I am planning to use -K as new option(available ones were
d,g,j,k,l,m,p,q,y).

I am planning to keep option same for pg_dumpall, as pg_dumpall internally
calls pg_dump with the options supplied by user.
In fact, I think we can hack the string passed to pg_dump to change the
option from -C to -K, but I am not able see if it will be way better than
using -K for both.

Suggestions?

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Amit Kapila
amit.kapila@huawei.com
In reply to: Amit Kapila (#17)
1 attachment(s)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

Tuesday, February 19, 2013 6:23 PM Amit Kapila wrote:
On Monday, February 18, 2013 1:41 PM Heikki Linnakangas wrote:

On 18.02.2013 06:07, Amit Kapila wrote:

On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote:

On Sun, Feb 17, 2013 at 1:35 AM, Amit

kapila<amit.kapila@huawei.com>

wrote:

Now the patch of Phil Sober provides 2 new API's

PQconninfoParseParams(), and PQconninfodefaultsMerge(),

using these API's I can think of below way for patch "pass a

connection string to pg_basebackup, ..."

1. Call existing function PQconinfoParse() with connection

string

input by user and get PQconninfoOption.

2. Now use the existing keywords (individual options specified

by

user) and extract the keywords from

PQconninfoOption structure and call new API

PQconninfoParseParams() which will return PQconninfoOption.

The PQconninfoOption structure returned in this step will

contain

all keywords

3. Call PQconninfodefaultsMerge() to merge any default values

if

exist. Not sure if this step is required?

4. Extract individual keywords from PQconninfoOption structure

and

call PQconnectdbParams.

Is this inline with what you have in mind or you have thought

of

some

other simpler way of using new API's?

Yep, that's roughly what I had in mind. I don't think it's

necessary

to

merge defaults in step 3, but it needs to add the

"replication=true"

and
"dbname=replication" options.

I could see the advantage of calling PQconninfoParseParams() in step-

2

is
that
it will remove the duplicate values by overriding the values for
conflicting
keywords.
This is done in function conninfo_array_parse() which is called from
PQconninfoParseParams().
Am I right or there is any other advantage of calling
PQconninfoParseParams()?

If there is no other advantage then this is done in

PQconnectdbParams()

also, so can't we avoid calling PQconninfoParseParams()?

----

I note that pg_dumpall also has a similar issue as pg_basebackup and
pg_receivexlog; there's no way to pass a connection string to it
either.

I think not only pg_dumpall, but we need to add it to pg_dump.
As -C is already used option in pg_dump, I need to use something
different.
I am planning to use -K as new option(available ones were
d,g,j,k,l,m,p,q,y).

I am planning to keep option same for pg_dumpall, as pg_dumpall
internally
calls pg_dump with the options supplied by user.
In fact, I think we can hack the string passed to pg_dump to change the
option from -C to -K, but I am not able see if it will be way better
than
using -K for both.

The patch for providing connection string for pg_basebackup, pg_receivexlog,
pg_dump and pg_dumpall is attached with this mail.

With Regards,
Amit Kapila.

Attachments:

pg_basebkup_recvxlog_dump_conn_str_v2.patchapplication/octet-stream; name=pg_basebkup_recvxlog_dump_conn_str_v2.patchDownload
*** a/doc/src/sgml/ref/pg_basebackup.sgml
--- b/doc/src/sgml/ref/pg_basebackup.sgml
***************
*** 357,362 **** PostgreSQL documentation
--- 357,375 ----
     <para>
      The following command-line options control the database connection parameters.
  
+ 	<variablelist>
+      <varlistentry>
+       <term><option>-C <replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <term><option>--connection-string=<replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <listitem>
+        <para>
+         Specifies connection string options, used for connecting to server.
+         These options can be used along with other user supplied options.
+         In case of conflicting option, user supplied option is choosen.
+        </para>
+       </listitem>
+      </varlistentry>
+ 
      <variablelist>
       <varlistentry>
        <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
*** a/doc/src/sgml/ref/pg_dump.sgml
--- b/doc/src/sgml/ref/pg_dump.sgml
***************
*** 815,820 **** PostgreSQL documentation
--- 815,833 ----
  
     <para>
      The following command-line options control the database connection parameters.
+     
+     <variablelist>
+      <varlistentry>
+       <term><option>-K <replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <term><option>--connection-string=<replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <listitem>
+        <para>
+         Specifies connection string options, used for connecting to server.
+         These options can be used along with other user supplied options.
+         In case of conflicting option, user supplied option is choosen.
+        </para>
+       </listitem>
+      </varlistentry>
  
      <variablelist>
       <varlistentry>
*** a/doc/src/sgml/ref/pg_dumpall.sgml
--- b/doc/src/sgml/ref/pg_dumpall.sgml
***************
*** 403,408 **** PostgreSQL documentation
--- 403,421 ----
  
    <para>
     The following command-line options control the database connection parameters.
+    
+    <variablelist>
+      <varlistentry>
+       <term><option>-K <replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <term><option>--connection-string=<replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <listitem>
+        <para>
+         Specifies connection string options, used for connecting to server.
+         These options can be used along with other user supplied options.
+         In case of conflicting option, user supplied option is choosen.
+        </para>
+       </listitem>
+      </varlistentry>
  
     <variablelist>
       <varlistentry>
*** a/doc/src/sgml/ref/pg_receivexlog.sgml
--- b/doc/src/sgml/ref/pg_receivexlog.sgml
***************
*** 121,126 **** PostgreSQL documentation
--- 121,139 ----
     <para>
      The following command-line options control the database connection parameters.
  
+ 	<variablelist>
+      <varlistentry>
+       <term><option>-C <replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <term><option>--connection-string=<replaceable class="parameter">"OPTIONS"</replaceable></option></term>
+       <listitem>
+        <para>
+         Specifies connection string options, used for connecting to server.
+         These options can be used along with other user supplied options.
+         In case of conflicting option, user supplied option is choosen.
+        </para>
+       </listitem>
+      </varlistentry>
+ 
      <variablelist>
       <varlistentry>
        <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
*** a/src/bin/pg_basebackup/pg_basebackup.c
--- b/src/bin/pg_basebackup/pg_basebackup.c
***************
*** 126,131 **** usage(void)
--- 126,132 ----
  	printf(_("  -V, --version          output version information, then exit\n"));
  	printf(_("  -?, --help             show this help, then exit\n"));
  	printf(_("\nConnection options:\n"));
+ 	printf(_("  -C, --connection-string=\"OPTIONS\" connect to server using OPTIONS\n"));
  	printf(_("  -h, --host=HOSTNAME    database server host or socket directory\n"));
  	printf(_("  -p, --port=PORT        database server port number\n"));
  	printf(_("  -s, --status-interval=INTERVAL\n"
***************
*** 1534,1539 **** main(int argc, char **argv)
--- 1535,1541 ----
  		{"pgdata", required_argument, NULL, 'D'},
  		{"format", required_argument, NULL, 'F'},
  		{"checkpoint", required_argument, NULL, 'c'},
+ 		{"connection-string", required_argument, NULL, 'C'},
  		{"write-recovery-conf", no_argument, NULL, 'R'},
  		{"xlog", no_argument, NULL, 'x'},
  		{"xlog-method", required_argument, NULL, 'X'},
***************
*** 1572,1578 **** main(int argc, char **argv)
  		}
  	}
  
! 	while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
  							long_options, &option_index)) != -1)
  	{
  		switch (c)
--- 1574,1580 ----
  		}
  	}
  
! 	while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:C:c:h:p:U:s:wWvP",
  							long_options, &option_index)) != -1)
  	{
  		switch (c)
***************
*** 1663,1668 **** main(int argc, char **argv)
--- 1665,1673 ----
  					exit(1);
  				}
  				break;
+ 			case 'C':
+ 				connection_string = pg_strdup(optarg);
+ 				break;
  			case 'h':
  				dbhost = pg_strdup(optarg);
  				break;
*** a/src/bin/pg_basebackup/pg_receivexlog.c
--- b/src/bin/pg_basebackup/pg_receivexlog.c
***************
*** 58,63 **** usage(void)
--- 58,64 ----
  	printf(_("  -V, --version          output version information, then exit\n"));
  	printf(_("  -?, --help             show this help, then exit\n"));
  	printf(_("\nConnection options:\n"));
+ 	printf(_("  -C, --connection-string=\"OPTIONS\" connect to server using OPTIONS\n"));
  	printf(_("  -h, --host=HOSTNAME    database server host or socket directory\n"));
  	printf(_("  -p, --port=PORT        database server port number\n"));
  	printf(_("  -s, --status-interval=INTERVAL\n"
***************
*** 305,310 **** main(int argc, char **argv)
--- 306,312 ----
  	static struct option long_options[] = {
  		{"help", no_argument, NULL, '?'},
  		{"version", no_argument, NULL, 'V'},
+ 		{"connection-string", required_argument, NULL, 'C'},
  		{"directory", required_argument, NULL, 'D'},
  		{"host", required_argument, NULL, 'h'},
  		{"port", required_argument, NULL, 'p'},
***************
*** 338,348 **** main(int argc, char **argv)
  		}
  	}
  
! 	while ((c = getopt_long(argc, argv, "D:h:p:U:s:nwWv",
  							long_options, &option_index)) != -1)
  	{
  		switch (c)
  		{
  			case 'D':
  				basedir = pg_strdup(optarg);
  				break;
--- 340,353 ----
  		}
  	}
  
! 	while ((c = getopt_long(argc, argv, "C:D:h:p:U:s:nwWv",
  							long_options, &option_index)) != -1)
  	{
  		switch (c)
  		{
+ 			case 'C':
+ 				connection_string = pg_strdup(optarg);
+ 				break;
  			case 'D':
  				basedir = pg_strdup(optarg);
  				break;
*** a/src/bin/pg_basebackup/streamutil.c
--- b/src/bin/pg_basebackup/streamutil.c
***************
*** 24,29 **** char	   *dbport = NULL;
--- 24,31 ----
  int			dbgetpassword = 0;	/* 0=auto, -1=never, 1=always */
  static char *dbpassword = NULL;
  PGconn	   *conn = NULL;
+ char	   *connection_string = NULL;
+ 
  
  /*
   * Connect to the server. Returns a valid PGconn pointer if connected,
***************
*** 34,64 **** PGconn *
  GetConnection(void)
  {
  	PGconn	   *tmpconn;
! 	int			argcount = 4;	/* dbname, replication, fallback_app_name,
! 								 * password */
! 	int			i;
  	const char **keywords;
  	const char **values;
  	char	   *password = NULL;
  	const char *tmpparam;
  
- 	if (dbhost)
- 		argcount++;
- 	if (dbuser)
- 		argcount++;
- 	if (dbport)
- 		argcount++;
- 
- 	keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
- 	values = pg_malloc0((argcount + 1) * sizeof(*values));
- 
- 	keywords[0] = "dbname";
- 	values[0] = "replication";
- 	keywords[1] = "replication";
- 	values[1] = "true";
- 	keywords[2] = "fallback_application_name";
- 	values[2] = progname;
- 	i = 3;
  	if (dbhost)
  	{
  		keywords[i] = "host";
--- 36,97 ----
  GetConnection(void)
  {
  	PGconn	   *tmpconn;
! 	int			i = 0;
  	const char **keywords;
  	const char **values;
  	char	   *password = NULL;
  	const char *tmpparam;
+ 	PQconninfoOption *conn_opts = NULL,
+ 			   *conn_opts_merged = NULL;
+ 	PQconninfoOption *conn_opt;
+ 	char	   *err_msg = NULL;
+ 
+ 
+ 	keywords = pg_malloc0(MAX_CONNINFO_OPTIONS * sizeof(*keywords));
+ 	values = pg_malloc0(MAX_CONNINFO_OPTIONS * sizeof(*values));
+ 
+ 	/*
+ 	 * Merge the connection info inputs given in form of connection string,
+ 	 * options and default values (dbname=replication, replication=true,
+ 	 * etc.). We could have written new function to connect using
+ 	 * PQconninfoOption, but it's not a lot of code to iterate through the
+ 	 * PGconnInfoOption array and turn it back into keywords and values
+ 	 * arrays, so just do that straight in the client code.
+ 	 */
+ 
+ 	if (connection_string)
+ 	{
+ 		conn_opts = PQconninfoParse(connection_string, &err_msg);
+ 		if (conn_opts != NULL)
+ 		{
+ 			for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
+ 			{
+ 				if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
+ 				{
+ 					keywords[i] = conn_opt->keyword;
+ 					values[i] = conn_opt->val;
+ 					i++;
+ 				}
+ 			}
+ 		}
+ 		else
+ 		{
+ 			fprintf(stderr, "%s: %s\n", progname, err_msg);
+ 			return NULL;
+ 		}
+ 
+ 	}
+ 
+ 	keywords[i] = "dbname";
+ 	values[i] = "replication";
+ 	i++;
+ 	keywords[i] = "replication";
+ 	values[i] = "true";
+ 	i++;
+ 	keywords[i] = "fallback_application_name";
+ 	values[i] = progname;
+ 	i++;
  
  	if (dbhost)
  	{
  		keywords[i] = "host";
***************
*** 78,83 **** GetConnection(void)
--- 111,143 ----
  		i++;
  	}
  
+ 	if (connection_string)
+ 	{
+ 		conn_opts_merged = PQconninfoParseParams(keywords, values,
+ 												 true, &err_msg);
+ 		if (conn_opts_merged != NULL)
+ 		{
+ 			i = 0;
+ 			MemSet(keywords, 0, MAX_CONNINFO_OPTIONS * sizeof(*keywords));
+ 			MemSet(values, 0, MAX_CONNINFO_OPTIONS * sizeof(*values));
+ 			for (conn_opt = conn_opts_merged; conn_opt->keyword != NULL; conn_opt++)
+ 			{
+ 				if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
+ 				{
+ 					keywords[i] = conn_opt->keyword;
+ 					values[i] = conn_opt->val;
+ 					i++;
+ 				}
+ 			}
+ 		}
+ 		else
+ 		{
+ 			fprintf(stderr, "%s: %s\n", progname, err_msg);
+ 			PQconninfoFree(conn_opts);
+ 			return NULL;
+ 		}
+ 	}
+ 
  	while (true)
  	{
  		if (password)
***************
*** 90,104 **** GetConnection(void)
  			 * meaning this is the call for a second session to the same
  			 * database, so just forcibly reuse that password.
  			 */
! 			keywords[argcount - 1] = "password";
! 			values[argcount - 1] = dbpassword;
  			dbgetpassword = -1; /* Don't try again if this fails */
  		}
  		else if (dbgetpassword == 1)
  		{
  			password = simple_prompt(_("Password: "), 100, false);
! 			keywords[argcount - 1] = "password";
! 			values[argcount - 1] = password;
  		}
  
  		tmpconn = PQconnectdbParams(keywords, values, true);
--- 150,164 ----
  			 * meaning this is the call for a second session to the same
  			 * database, so just forcibly reuse that password.
  			 */
! 			keywords[i] = "password";
! 			values[i] = dbpassword;
  			dbgetpassword = -1; /* Don't try again if this fails */
  		}
  		else if (dbgetpassword == 1)
  		{
  			password = simple_prompt(_("Password: "), 100, false);
! 			keywords[i] = "password";
! 			values[i] = password;
  		}
  
  		tmpconn = PQconnectdbParams(keywords, values, true);
***************
*** 130,141 **** GetConnection(void)
--- 190,205 ----
  			PQfinish(tmpconn);
  			free(values);
  			free(keywords);
+ 			PQconninfoFree(conn_opts);
+ 			PQconninfoFree(conn_opts_merged);
  			return NULL;
  		}
  
  		/* Connection ok! */
  		free(values);
  		free(keywords);
+ 		PQconninfoFree(conn_opts);
+ 		PQconninfoFree(conn_opts_merged);
  
  		/*
  		 * Ensure we have the same value of integer timestamps as the server
*** a/src/bin/pg_basebackup/streamutil.h
--- b/src/bin/pg_basebackup/streamutil.h
***************
*** 5,10 **** extern char *dbhost;
--- 5,11 ----
  extern char *dbuser;
  extern char *dbport;
  extern int	dbgetpassword;
+ extern char *connection_string;
  
  /* Connection kept global so we can disconnect easily */
  extern PGconn *conn;
*** a/src/bin/pg_dump/dumputils.c
--- b/src/bin/pg_dump/dumputils.c
***************
*** 28,33 **** extern const int NumFEScanKeywords;
--- 28,34 ----
  /* Globals exported by this file */
  int			quote_all_identifiers = 0;
  const char *progname = NULL;
+ char	   *connection_string = NULL;
  
  #define MAX_ON_EXIT_NICELY				20
  
*** a/src/bin/pg_dump/dumputils.h
--- b/src/bin/pg_dump/dumputils.h
***************
*** 44,49 **** typedef void (*on_exit_nicely_callback) (int code, void *arg);
--- 44,50 ----
  
  extern int	quote_all_identifiers;
  extern const char *progname;
+ extern char *connection_string;
  
  extern void init_parallel_dump_utils(void);
  extern const char *fmtId(const char *identifier);
*** a/src/bin/pg_dump/pg_backup_db.c
--- b/src/bin/pg_dump/pg_backup_db.c
***************
*** 254,283 **** ConnectDatabase(Archive *AHX,
  	 */
  	do
  	{
! #define PARAMS_ARRAY_SIZE	7
! 		const char **keywords = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*keywords));
! 		const char **values = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*values));
  
- 		keywords[0] = "host";
- 		values[0] = pghost;
- 		keywords[1] = "port";
- 		values[1] = pgport;
- 		keywords[2] = "user";
- 		values[2] = username;
- 		keywords[3] = "password";
- 		values[3] = password;
- 		keywords[4] = "dbname";
- 		values[4] = dbname;
- 		keywords[5] = "fallback_application_name";
- 		values[5] = progname;
- 		keywords[6] = NULL;
- 		values[6] = NULL;
  
  		new_pass = false;
  		AH->connection = PQconnectdbParams(keywords, values, true);
  
  		free(keywords);
  		free(values);
  
  		if (!AH->connection)
  			exit_horribly(modulename, "failed to connect to database\n");
--- 254,349 ----
  	 */
  	do
  	{
! 		const char **keywords = pg_malloc(MAX_CONNINFO_OPTIONS * sizeof(*keywords));
! 		const char **values = pg_malloc(MAX_CONNINFO_OPTIONS * sizeof(*values));
! 		PQconninfoOption *conn_opts = NULL,
! 				   *conn_opts_merged = NULL;
! 		PQconninfoOption *conn_opt;
! 		char	   *err_msg = NULL;
! 		int			i = 0;
! 
! 		/*
! 		 * Merge the connection info inputs given in form of connection string
! 		 * and options. We could have written new function to connect using
! 		 * PQconninfoOption, but it's not a lot of code to iterate through the
! 		 * PGconnInfoOption array and turn it back into keywords and values
! 		 * arrays, so just do that straight in the client code.
! 		 */
! 		if (connection_string)
! 		{
! 			conn_opts = PQconninfoParse(connection_string, &err_msg);
! 			if (conn_opts != NULL)
! 			{
! 				for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
! 				{
! 					if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
! 					{
! 						keywords[i] = conn_opt->keyword;
! 						values[i] = conn_opt->val;
! 						i++;
! 					}
! 				}
! 			}
! 			else
! 				exit_horribly(modulename, "%s\n", err_msg);
! 		}
! 
! 		keywords[i] = "host";
! 		values[i] = pghost;
! 		i++;
! 		keywords[i] = "port";
! 		values[i] = pgport;
! 		i++;
! 		keywords[i] = "user";
! 		values[i] = username;
! 		i++;
! 		keywords[i] = "password";
! 		values[i] = password;
! 		i++;
! 		keywords[i] = "dbname";
! 		values[i] = dbname;
! 		i++;
! 		keywords[i] = "fallback_application_name";
! 		values[i] = progname;
! 		i++;
! 		keywords[i] = NULL;
! 		values[i] = NULL;
! 
! 		if (connection_string)
! 		{
! 			conn_opts_merged = PQconninfoParseParams(keywords, values,
! 													 true, &err_msg);
! 			if (conn_opts_merged != NULL)
! 			{
! 				i = 0;
! 				MemSet(keywords, 0, MAX_CONNINFO_OPTIONS * sizeof(*keywords));
! 				MemSet(values, 0, MAX_CONNINFO_OPTIONS * sizeof(*values));
! 				for (conn_opt = conn_opts_merged; conn_opt->keyword != NULL; conn_opt++)
! 				{
! 					if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
! 					{
! 						keywords[i] = conn_opt->keyword;
! 						values[i] = conn_opt->val;
! 						i++;
! 					}
! 				}
! 			}
! 			else
! 			{
! 				PQconninfoFree(conn_opts);
! 				exit_horribly(modulename, "%s\n", err_msg);
! 
! 			}
! 		}
  
  
  		new_pass = false;
  		AH->connection = PQconnectdbParams(keywords, values, true);
  
  		free(keywords);
  		free(values);
+ 		PQconninfoFree(conn_opts);
+ 		PQconninfoFree(conn_opts_merged);
  
  		if (!AH->connection)
  			exit_horribly(modulename, "failed to connect to database\n");
*** a/src/bin/pg_dump/pg_dump.c
--- b/src/bin/pg_dump/pg_dump.c
***************
*** 311,316 **** main(int argc, char **argv)
--- 311,317 ----
  		{"format", required_argument, NULL, 'F'},
  		{"host", required_argument, NULL, 'h'},
  		{"ignore-version", no_argument, NULL, 'i'},
+ 		{"connection-string", required_argument, NULL, 'K'},
  		{"no-reconnect", no_argument, NULL, 'R'},
  		{"oids", no_argument, NULL, 'o'},
  		{"no-owner", no_argument, NULL, 'O'},
***************
*** 387,393 **** main(int argc, char **argv)
  		}
  	}
  
! 	while ((c = getopt_long(argc, argv, "abcCE:f:F:h:in:N:oOp:RsS:t:T:U:vwWxZ:",
  							long_options, &optindex)) != -1)
  	{
  		switch (c)
--- 388,394 ----
  		}
  	}
  
! 	while ((c = getopt_long(argc, argv, "abcCE:f:F:h:iK:n:N:oOp:RsS:t:T:U:vwWxZ:",
  							long_options, &optindex)) != -1)
  	{
  		switch (c)
***************
*** 428,433 **** main(int argc, char **argv)
--- 429,438 ----
  				/* ignored, deprecated option */
  				break;
  
+ 			case 'K':
+ 				connection_string = pg_strdup(optarg);
+ 				break;
+ 
  			case 'n':			/* include schema(s) */
  				simple_string_list_append(&schema_include_patterns, optarg);
  				include_everything = false;
***************
*** 872,877 **** help(const char *progname)
--- 877,883 ----
  			 "                               ALTER OWNER commands to set ownership\n"));
  
  	printf(_("\nConnection options:\n"));
+ 	printf(_("  -K, --connection-string=\"OPTIONS\" connect to server using OPTIONS\n"));
  	printf(_("  -h, --host=HOSTNAME      database server host or socket directory\n"));
  	printf(_("  -p, --port=PORT          database server port number\n"));
  	printf(_("  -U, --username=NAME      connect as specified database user\n"));
*** a/src/bin/pg_dump/pg_dumpall.c
--- b/src/bin/pg_dump/pg_dumpall.c
***************
*** 79,85 **** static int	server_version;
  static FILE *OPF;
  static char *filename = NULL;
  
- 
  int
  main(int argc, char *argv[])
  {
--- 79,84 ----
***************
*** 90,95 **** main(int argc, char *argv[])
--- 89,95 ----
  		{"globals-only", no_argument, NULL, 'g'},
  		{"host", required_argument, NULL, 'h'},
  		{"ignore-version", no_argument, NULL, 'i'},
+ 		{"connection-string", required_argument, NULL, 'K'},
  		{"database", required_argument, NULL, 'l'},
  		{"oids", no_argument, NULL, 'o'},
  		{"no-owner", no_argument, NULL, 'O'},
***************
*** 187,193 **** main(int argc, char *argv[])
  
  	pgdumpopts = createPQExpBuffer();
  
! 	while ((c = getopt_long(argc, argv, "acf:gh:il:oOp:rsS:tU:vwWx", long_options, &optindex)) != -1)
  	{
  		switch (c)
  		{
--- 187,193 ----
  
  	pgdumpopts = createPQExpBuffer();
  
! 	while ((c = getopt_long(argc, argv, "acf:gh:iK:l:oOp:rsS:tU:vwWx", long_options, &optindex)) != -1)
  	{
  		switch (c)
  		{
***************
*** 220,225 **** main(int argc, char *argv[])
--- 220,231 ----
  				/* ignored, deprecated option */
  				break;
  
+ 			case 'K':
+ 				connection_string = pg_strdup(optarg);
+ 				appendPQExpBuffer(pgdumpopts, " -K ");
+ 				doShellQuoting(pgdumpopts, connection_string);
+ 				break;
+ 
  			case 'l':
  				pgdb = pg_strdup(optarg);
  				break;
***************
*** 567,572 **** help(void)
--- 573,579 ----
  			 "                               ALTER OWNER commands to set ownership\n"));
  
  	printf(_("\nConnection options:\n"));
+ 	printf(_("  -K, --connection-string=\"OPTIONS\" connect to server using OPTIONS\n"));
  	printf(_("  -h, --host=HOSTNAME      database server host or socket directory\n"));
  	printf(_("  -l, --database=DBNAME    alternative default database\n"));
  	printf(_("  -p, --port=PORT          database server port number\n"));
***************
*** 1710,1739 **** connectDatabase(const char *dbname, const char *pghost, const char *pgport,
  	 */
  	do
  	{
! #define PARAMS_ARRAY_SIZE	7
! 		const char **keywords = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*keywords));
! 		const char **values = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*values));
! 
! 		keywords[0] = "host";
! 		values[0] = pghost;
! 		keywords[1] = "port";
! 		values[1] = pgport;
! 		keywords[2] = "user";
! 		values[2] = pguser;
! 		keywords[3] = "password";
! 		values[3] = password;
! 		keywords[4] = "dbname";
! 		values[4] = dbname;
! 		keywords[5] = "fallback_application_name";
! 		values[5] = progname;
! 		keywords[6] = NULL;
! 		values[6] = NULL;
  
  		new_pass = false;
  		conn = PQconnectdbParams(keywords, values, true);
  
  		free(keywords);
  		free(values);
  
  		if (!conn)
  		{
--- 1717,1814 ----
  	 */
  	do
  	{
! 		const char **keywords = pg_malloc(MAX_CONNINFO_OPTIONS * sizeof(*keywords));
! 		const char **values = pg_malloc(MAX_CONNINFO_OPTIONS * sizeof(*values));
! 		PQconninfoOption *conn_opts = NULL,
! 				   *conn_opts_merged = NULL;
! 		PQconninfoOption *conn_opt;
! 		char	   *err_msg = NULL;
! 		int			i = 0;
! 
! 		/*
! 		 * Merge the connection info inputs given in form of connection string
! 		 * and options. We could have written new function to connect using
! 		 * PQconninfoOption, but it's not a lot of code to iterate through the
! 		 * PGconnInfoOption array and turn it back into keywords and values
! 		 * arrays, so just do that straight in the client code.
! 		 */
! 		if (connection_string)
! 		{
! 			conn_opts = PQconninfoParse(connection_string, &err_msg);
! 			if (conn_opts != NULL)
! 			{
! 				for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
! 				{
! 					if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
! 					{
! 						keywords[i] = conn_opt->keyword;
! 						values[i] = conn_opt->val;
! 						i++;
! 					}
! 				}
! 			}
! 			else
! 			{
! 				fprintf(stderr, "%s: %s\n", progname, err_msg);
! 				return NULL;
! 			}
! 		}
! 
! 		keywords[i] = "host";
! 		values[i] = pghost;
! 		i++;
! 		keywords[i] = "port";
! 		values[i] = pgport;
! 		i++;
! 		keywords[i] = "user";
! 		values[i] = pguser;
! 		i++;
! 		keywords[i] = "password";
! 		values[i] = password;
! 		i++;
! 		keywords[i] = "dbname";
! 		values[i] = dbname;
! 		i++;
! 		keywords[i] = "fallback_application_name";
! 		values[i] = progname;
! 		i++;
! 		keywords[i] = NULL;
! 		values[i] = NULL;
! 
! 		if (connection_string)
! 		{
! 			conn_opts_merged = PQconninfoParseParams(keywords, values,
! 													 true, &err_msg);
! 			if (conn_opts_merged != NULL)
! 			{
! 				i = 0;
! 				MemSet(keywords, 0, MAX_CONNINFO_OPTIONS * sizeof(*keywords));
! 				MemSet(values, 0, MAX_CONNINFO_OPTIONS * sizeof(*values));
! 				for (conn_opt = conn_opts_merged; conn_opt->keyword != NULL; conn_opt++)
! 				{
! 					if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
! 					{
! 						keywords[i] = conn_opt->keyword;
! 						values[i] = conn_opt->val;
! 						i++;
! 					}
! 				}
! 			}
! 			else
! 			{
! 				fprintf(stderr, "%s: %s\n", progname, err_msg);
! 				PQconninfoFree(conn_opts);
! 				return NULL;
! 			}
! 		}
  
  		new_pass = false;
  		conn = PQconnectdbParams(keywords, values, true);
  
  		free(keywords);
  		free(values);
+ 		PQconninfoFree(conn_opts);
+ 		PQconninfoFree(conn_opts_merged);
  
  		if (!conn)
  		{
*** a/src/include/pg_config_manual.h
--- b/src/include/pg_config_manual.h
***************
*** 200,205 ****
--- 200,213 ----
  #endif
  
  /*
+  * MAX_CONNINFO_OPTIONS: maximum number of options for connection.
+  *				It refers to PQconninfoOptions, currently it is less than 50, kept
+  *				this considering future additions, however it can be kept same.
+  */
+ #define MAX_CONNINFO_OPTIONS		50
+ 
+ 
+ /*
   *------------------------------------------------------------------------
   * The following symbols are for enabling debugging code, not for
   * controlling user-visible features or resource limits.
*** a/src/interfaces/libpq/exports.txt
--- b/src/interfaces/libpq/exports.txt
***************
*** 165,167 **** lo_lseek64                162
--- 165,169 ----
  lo_tell64                 163
  lo_truncate64             164
  PQconninfo                165
+ PQconninfoParseParams     166
+ PQconninfodefaultsMerge   167
*** a/src/interfaces/libpq/fe-connect.c
--- b/src/interfaces/libpq/fe-connect.c
***************
*** 4002,4007 **** PQconninfoParse(const char *conninfo, char **errmsg)
--- 4002,4086 ----
  }
  
  /*
+  *		PQconninfoParseParams
+  *
+  * Parse a string like PQconnectdbParams() would do and return the
+  * resulting connection options array.	NULL is returned on failure.
+  * The result contains only options specified directly in the array,
+  * not any possible default values.
+  *
+  * If errmsg isn't NULL, *errmsg is set to NULL on success, or a malloc'd
+  * string on failure (use PQfreemem to free it).  In out-of-memory conditions
+  * both *errmsg and the result could be NULL.
+  *
+  * NOTE: the returned array is dynamically allocated and should
+  * be freed when no longer needed via PQconninfoFree().
+  */
+ PQconninfoOption *
+ PQconninfoParseParams(const char *const * keywords,
+ 					  const char *const * values, int expand_dbname,
+ 					  char **errmsg)
+ {
+ 	PQExpBufferData errorBuf;
+ 	PQconninfoOption *connOptions;
+ 
+ 	if (errmsg)
+ 		*errmsg = NULL;			/* default */
+ 	initPQExpBuffer(&errorBuf);
+ 	if (PQExpBufferDataBroken(errorBuf))
+ 		return NULL;			/* out of memory already :-( */
+ 	connOptions = conninfo_array_parse(keywords, values,
+ 									   &errorBuf,
+ 									   false, expand_dbname);
+ 
+ 	if (connOptions == NULL && errmsg)
+ 		*errmsg = errorBuf.data;
+ 	else
+ 		termPQExpBuffer(&errorBuf);
+ 	return connOptions;
+ }
+ 
+ /*
+  *		PQconninfodefaultsMerge
+  *
+  * Add the default values for any unspecified options to the connection
+  * options array.
+  *
+  * Defaults are obtained from a service file, environment variables, etc.
+  *
+  * Returns 1 if successful, otherwise 0; connOptions is freed and errmsg is
+  * filled in upon failure.  Note that failure to locate a default value is
+  * not an error condition here --- we just leave the option's value as NULL.
+  *
+  * If errmsg isn't NULL, *errmsg is set to NULL on success, or a malloc'd
+  * string on failure (use PQfreemem to free it).  In out-of-memory conditions
+  * both *errmsg and the result could be NULL.
+  */
+ int
+ PQconninfodefaultsMerge(PQconninfoOption *connOptions, char **errmsg)
+ {
+ 	PQExpBufferData errorBuf;
+ 
+ 	if (errmsg)
+ 		*errmsg = NULL;
+ 	initPQExpBuffer(&errorBuf);
+ 	if (PQExpBufferDataBroken(errorBuf))
+ 	{
+ 		PQconninfoFree(connOptions);
+ 		return 0;
+ 	}
+ 	if (!conninfo_add_defaults(connOptions, &errorBuf))
+ 	{
+ 		if (errmsg)
+ 			*errmsg = errorBuf.data;
+ 		PQconninfoFree(connOptions);
+ 		return 0;
+ 	}
+ 	termPQExpBuffer(&errorBuf);
+ 	return 1;
+ }
+ 
+ /*
   * Build a working copy of the constant PQconninfoOptions array.
   */
  static PQconninfoOption *
*** a/src/interfaces/libpq/libpq-fe.h
--- b/src/interfaces/libpq/libpq-fe.h
***************
*** 267,276 **** extern PQconninfoOption *PQconndefaults(void);
  /* parse connection options in same way as PQconnectdb */
  extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
  
  /* return the connection options used by a live connection */
  extern PQconninfoOption *PQconninfo(PGconn *conn);
  
! /* free the data structure returned by PQconndefaults() or PQconninfoParse() */
  extern void PQconninfoFree(PQconninfoOption *connOptions);
  
  /*
--- 267,284 ----
  /* parse connection options in same way as PQconnectdb */
  extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
  
+ /* parse connection options in same way as PQconnectdbParams */
+ extern PQconninfoOption *PQconninfoParseParams(const char *const * keywords,
+ 					  const char *const * values, int expand_dbname,
+ 					  char **errmsg);
+ 
+ /* Merge default connection options into an existing PQconninfoOption struct */
+ extern int	PQconninfodefaultsMerge(PQconninfoOption *connOptions, char **errmsg);
+ 
  /* return the connection options used by a live connection */
  extern PQconninfoOption *PQconninfo(PGconn *conn);
  
! /* free the data structure returned by PQconndefaults(), PQconninfoParse(), or PQconninfoParseParams() */
  extern void PQconninfoFree(PQconninfoOption *connOptions);
  
  /*
#19Heikki Linnakangas
hlinnakangas@vmware.com
In reply to: Amit Kapila (#18)
3 attachment(s)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On 20.02.2013 11:42, Amit Kapila wrote:

The patch for providing connection string for pg_basebackup, pg_receivexlog,
pg_dump and pg_dumpall is attached with this mail.

Thanks. Now that I look at this patch, I realize that we don't actually
need these new functions for pg_basebackup and friends after all. We
already have PQconninfoParse(), we can just use that.

pg_dump can already take a connection string:

pg_dump "dbname=postgres port=5432"

For consistency with psql and other tools, perhaps we should add a "-d"
option to pg_dump, so that you could do:

pg_dump -d "dbname=postgres port=5432"

It'd be nice to call the option -d or --dbname in all the tools. That's
a bit confusing for pg_basebackup and pg_receivexlog, as it can *not*
actually be a database name, but it would be otherwise consistent with
the other commands.

I came up with the attached three patches. The first adds -d/--dbname
option to pg_basebackup and pg_receivexlog. The second adds it to
pg_dump, per above. The third adds it to pg_dumpall.

The third patch is a bit complicated. It first parses the user-specified
connection string using PQconninfoParse, so that it can merge in some
extra keywords: user, host, password, dbname and
fallback_application_name. It then calls PQconnectdbParams with the
keyword/value pairs. After making the initial connection to postgres or
template1 database, it calls PQconninfo() to again extract the
keyword/value pairs in effect in the connection, and constructs a new
connection string from them. That new connection string is then passed
to pg_dump on the command line, with the database name appended to it.

That seems to work, although it's perhaps a bit Rube Goldbergian. One
step of deparsing and parsing could be avoided by keeping the
keyword/value pairs from the first PQconninfoParse() call, instead of
constructing them again with PQconninfo(). I'll experiment with that
tomorrow.

The docs need some improvement. In those commands where you can't pass a
database name to the -d/--dbname option, only a connection string, I
kept your wording in the docs. But it ought to explain the seemingly
strange name for the option, and more. I'll take another whack at that
tomorrow as well.

Where does this leave the PQconninfoParseParams/PQconninfodefaultsMerge
patch? I'm not sure. Somehow I thought it would be necessary for this
work, but it wasn't. I didn't remember that we already have
PQconninfoParse() function, which was enough. So, what's the use case
for those functions?

- Heikki

Attachments:

0001-Add-d-option-for-specifying-a-connection-string-to-p.patchtext/x-diff; name=0001-Add-d-option-for-specifying-a-connection-string-to-p.patchDownload
>From ebfcff54ae934b38f3bfecfbe8dbe0cbe0573c95 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 20 Feb 2013 20:49:24 +0200
Subject: [PATCH 1/3] Add -d option, for specifying a connection string, to
 pg_basebackup and pg_receivexlog.

It's a bit strange that the option is called -d/--dbname, when in fact
you can *not* pass a database name to it. But it's consistent with other
client tools, where you can pass a connection string using the -d option.
---
 doc/src/sgml/ref/pg_basebackup.sgml    |   12 +++++
 doc/src/sgml/ref/pg_receivexlog.sgml   |   12 +++++
 src/bin/pg_basebackup/pg_basebackup.c  |    7 ++-
 src/bin/pg_basebackup/pg_receivexlog.c |    7 ++-
 src/bin/pg_basebackup/streamutil.c     |   87 +++++++++++++++++++++++---------
 src/bin/pg_basebackup/streamutil.h     |    1 +
 6 files changed, 101 insertions(+), 25 deletions(-)

diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 2f89f2c..3ab460a 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -359,6 +359,18 @@ PostgreSQL documentation
 
     <variablelist>
      <varlistentry>
+      <term><option>-d <replaceable class="parameter">connstr</replaceable></option></term>
+      <term><option>--dbname=<replaceable class="parameter">connstr</replaceable></option></term>
+      <listitem>
+       <para>
+        Specifies connection string options, used for connecting to server.
+        These options can be used along with other user supplied options.
+        In case of aconflicting options, the user supplied option is used.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
       <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
       <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
       <listitem>
diff --git a/doc/src/sgml/ref/pg_receivexlog.sgml b/doc/src/sgml/ref/pg_receivexlog.sgml
index d06dd1f..314da0e 100644
--- a/doc/src/sgml/ref/pg_receivexlog.sgml
+++ b/doc/src/sgml/ref/pg_receivexlog.sgml
@@ -123,6 +123,18 @@ PostgreSQL documentation
 
     <variablelist>
      <varlistentry>
+      <term><option>-d <replaceable class="parameter">connstr</replaceable></option></term>
+      <term><option>--dbname=<replaceable class="parameter">connstr</replaceable></option></term>
+      <listitem>
+       <para>
+        Specifies connection string options, used for connecting to server.
+        These options can be used along with other user supplied options.
+        In case of conflicting options, the user supplied option is used.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
       <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
       <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
       <listitem>
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index fb5a1bd..2de03ac 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -126,6 +126,7 @@ usage(void)
 	printf(_("  -V, --version          output version information, then exit\n"));
 	printf(_("  -?, --help             show this help, then exit\n"));
 	printf(_("\nConnection options:\n"));
+	printf(_("  -d, --dbname=CONNSTR   connection string\n"));
 	printf(_("  -h, --host=HOSTNAME    database server host or socket directory\n"));
 	printf(_("  -p, --port=PORT        database server port number\n"));
 	printf(_("  -s, --status-interval=INTERVAL\n"
@@ -1540,6 +1541,7 @@ main(int argc, char **argv)
 		{"gzip", no_argument, NULL, 'z'},
 		{"compress", required_argument, NULL, 'Z'},
 		{"label", required_argument, NULL, 'l'},
+		{"dbname", required_argument, NULL, 'd'},
 		{"host", required_argument, NULL, 'h'},
 		{"port", required_argument, NULL, 'p'},
 		{"username", required_argument, NULL, 'U'},
@@ -1572,7 +1574,7 @@ main(int argc, char **argv)
 		}
 	}
 
-	while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
+	while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:d:c:h:p:U:s:wWvP",
 							long_options, &option_index)) != -1)
 	{
 		switch (c)
@@ -1663,6 +1665,9 @@ main(int argc, char **argv)
 					exit(1);
 				}
 				break;
+			case 'd':
+				connection_string = pg_strdup(optarg);
+				break;
 			case 'h':
 				dbhost = pg_strdup(optarg);
 				break;
diff --git a/src/bin/pg_basebackup/pg_receivexlog.c b/src/bin/pg_basebackup/pg_receivexlog.c
index 33dbc50..352ff35 100644
--- a/src/bin/pg_basebackup/pg_receivexlog.c
+++ b/src/bin/pg_basebackup/pg_receivexlog.c
@@ -58,6 +58,7 @@ usage(void)
 	printf(_("  -V, --version          output version information, then exit\n"));
 	printf(_("  -?, --help             show this help, then exit\n"));
 	printf(_("\nConnection options:\n"));
+	printf(_("  -d, --dbname=CONNSTR   connection string\n"));
 	printf(_("  -h, --host=HOSTNAME    database server host or socket directory\n"));
 	printf(_("  -p, --port=PORT        database server port number\n"));
 	printf(_("  -s, --status-interval=INTERVAL\n"
@@ -306,6 +307,7 @@ main(int argc, char **argv)
 		{"help", no_argument, NULL, '?'},
 		{"version", no_argument, NULL, 'V'},
 		{"directory", required_argument, NULL, 'D'},
+		{"dbname", required_argument, NULL, 'd'},
 		{"host", required_argument, NULL, 'h'},
 		{"port", required_argument, NULL, 'p'},
 		{"username", required_argument, NULL, 'U'},
@@ -338,7 +340,7 @@ main(int argc, char **argv)
 		}
 	}
 
-	while ((c = getopt_long(argc, argv, "D:h:p:U:s:nwWv",
+	while ((c = getopt_long(argc, argv, "D:d:h:p:U:s:nwWv",
 							long_options, &option_index)) != -1)
 	{
 		switch (c)
@@ -346,6 +348,9 @@ main(int argc, char **argv)
 			case 'D':
 				basedir = pg_strdup(optarg);
 				break;
+			case 'd':
+				connection_string = pg_strdup(optarg);
+				break;
 			case 'h':
 				dbhost = pg_strdup(optarg);
 				break;
diff --git a/src/bin/pg_basebackup/streamutil.c b/src/bin/pg_basebackup/streamutil.c
index 8a43c4b..a878dd4 100644
--- a/src/bin/pg_basebackup/streamutil.c
+++ b/src/bin/pg_basebackup/streamutil.c
@@ -18,6 +18,7 @@
 #include <string.h>
 
 const char *progname;
+char	   *connection_string = NULL;
 char	   *dbhost = NULL;
 char	   *dbuser = NULL;
 char	   *dbport = NULL;
@@ -34,31 +35,67 @@ PGconn *
 GetConnection(void)
 {
 	PGconn	   *tmpconn;
-	int			argcount = 4;	/* dbname, replication, fallback_app_name,
-								 * password */
+	int			argcount = 7;	/* dbname, replication, fallback_app_name,
+								 * host, user, port, password */
 	int			i;
 	const char **keywords;
 	const char **values;
 	char	   *password = NULL;
 	const char *tmpparam;
+	PQconninfoOption *conn_opts = NULL;
+	PQconninfoOption *conn_opt;
+	char	   *err_msg = NULL;
+
+	/*
+	 * Merge the connection info inputs given in form of connection string,
+	 * options and default values (dbname=replication, replication=true,
+	 * etc.)
+	 */
+	i = 0;
+	if (connection_string)
+	{
+		conn_opts = PQconninfoParse(connection_string, &err_msg);
+		if (conn_opts == NULL)
+		{
+			fprintf(stderr, "%s: %s\n", progname, err_msg);
+			return NULL;
+		}
+
+		for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
+		{
+			if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
+				argcount++;
+		}
+
+		keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
+		values = pg_malloc0((argcount + 1) * sizeof(*values));
+
+		for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
+		{
+			if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
+			{
+				keywords[i] = conn_opt->keyword;
+				values[i] = conn_opt->val;
+				i++;
+			}
+		}
+	}
+	else
+	{
+		keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
+		values = pg_malloc0((argcount + 1) * sizeof(*values));
+	}
+
+	keywords[i] = "dbname";
+	values[i] = "replication";
+	i++;
+	keywords[i] = "replication";
+	values[i] = "true";
+	i++;
+	keywords[i] = "fallback_application_name";
+	values[i] = progname;
+	i++;
 
-	if (dbhost)
-		argcount++;
-	if (dbuser)
-		argcount++;
-	if (dbport)
-		argcount++;
-
-	keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
-	values = pg_malloc0((argcount + 1) * sizeof(*values));
-
-	keywords[0] = "dbname";
-	values[0] = "replication";
-	keywords[1] = "replication";
-	values[1] = "true";
-	keywords[2] = "fallback_application_name";
-	values[2] = progname;
-	i = 3;
 	if (dbhost)
 	{
 		keywords[i] = "host";
@@ -90,15 +127,15 @@ GetConnection(void)
 			 * meaning this is the call for a second session to the same
 			 * database, so just forcibly reuse that password.
 			 */
-			keywords[argcount - 1] = "password";
-			values[argcount - 1] = dbpassword;
+			keywords[i] = "password";
+			values[i] = dbpassword;
 			dbgetpassword = -1; /* Don't try again if this fails */
 		}
 		else if (dbgetpassword == 1)
 		{
 			password = simple_prompt(_("Password: "), 100, false);
-			keywords[argcount - 1] = "password";
-			values[argcount - 1] = password;
+			keywords[i] = "password";
+			values[i] = password;
 		}
 
 		tmpconn = PQconnectdbParams(keywords, values, true);
@@ -130,12 +167,16 @@ GetConnection(void)
 			PQfinish(tmpconn);
 			free(values);
 			free(keywords);
+			if (conn_opts)
+				PQconninfoFree(conn_opts);
 			return NULL;
 		}
 
 		/* Connection ok! */
 		free(values);
 		free(keywords);
+		if (conn_opts)
+			PQconninfoFree(conn_opts);
 
 		/*
 		 * Ensure we have the same value of integer timestamps as the server
diff --git a/src/bin/pg_basebackup/streamutil.h b/src/bin/pg_basebackup/streamutil.h
index 4f5ff91..77d6b86 100644
--- a/src/bin/pg_basebackup/streamutil.h
+++ b/src/bin/pg_basebackup/streamutil.h
@@ -1,6 +1,7 @@
 #include "libpq-fe.h"
 
 extern const char *progname;
+extern char *connection_string;
 extern char *dbhost;
 extern char *dbuser;
 extern char *dbport;
-- 
1.7.10.4

0002-Add-d-option-to-pg_dump.patchtext/x-diff; name=0002-Add-d-option-to-pg_dump.patchDownload
>From 4a5633852134dd7bf8687f72f6fa6eaa00059952 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 20 Feb 2013 20:50:59 +0200
Subject: [PATCH 2/3] Add -d option to pg_dump.

You could already pass a database name on the command line, as the last
argument without any option character. But for consistency with psql and
other client tools, also accept it using -d/--dbname.
---
 doc/src/sgml/ref/pg_dump.sgml |   20 ++++++++++++++++++++
 src/bin/pg_dump/pg_dump.c     |   15 ++++++++++++---
 2 files changed, 32 insertions(+), 3 deletions(-)

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 152edcb..6d0f214 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -818,6 +818,26 @@ PostgreSQL documentation
 
     <variablelist>
      <varlistentry>
+      <term><option>-d <replaceable class="parameter">dbname</replaceable></></term>
+      <term><option>--dbname=<replaceable class="parameter">dbname</replaceable></></term>
+      <listitem>
+      <para>
+       Specifies the name of the database to connect to. This is
+       equivalent to specifying <replaceable
+       class="parameter">dbname</replaceable> as the first non-option
+       argument on the command line.
+      </para>
+      <para>
+       If this parameter contains an <symbol>=</symbol> sign or starts
+       with a valid <acronym>URI</acronym> prefix
+       (<literal>postgresql://</literal>
+       or <literal>postgres://</literal>), it is treated as a
+       <parameter>conninfo</parameter> string. See <xref linkend="libpq-connect"> for more information.
+      </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
       <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
       <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 43d571c..4923847 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -307,6 +307,7 @@ main(int argc, char **argv)
 		{"blobs", no_argument, NULL, 'b'},
 		{"clean", no_argument, NULL, 'c'},
 		{"create", no_argument, NULL, 'C'},
+		{"dbname", required_argument, NULL, 'd'},
 		{"file", required_argument, NULL, 'f'},
 		{"format", required_argument, NULL, 'F'},
 		{"host", required_argument, NULL, 'h'},
@@ -387,7 +388,7 @@ main(int argc, char **argv)
 		}
 	}
 
-	while ((c = getopt_long(argc, argv, "abcCE:f:F:h:in:N:oOp:RsS:t:T:U:vwWxZ:",
+	while ((c = getopt_long(argc, argv, "abcCd:E:f:F:h:iK:n:N:oOp:RsS:t:T:U:vwWxZ:",
 							long_options, &optindex)) != -1)
 	{
 		switch (c)
@@ -408,6 +409,10 @@ main(int argc, char **argv)
 				outputCreateDB = 1;
 				break;
 
+			case 'd':			/* database name */
+				dbname = pg_strdup(optarg);
+				break;
+
 			case 'E':			/* Dump encoding */
 				dumpencoding = pg_strdup(optarg);
 				break;
@@ -520,8 +525,11 @@ main(int argc, char **argv)
 		}
 	}
 
-	/* Get database name from command line */
-	if (optind < argc)
+	/*
+	 * Non-option argument specifies database name as long as it wasn't
+	 * already specified with -d / --dbname
+	 */
+	if (optind < argc && dbname == NULL)
 		dbname = argv[optind++];
 
 	/* Complain if any arguments remain */
@@ -872,6 +880,7 @@ help(const char *progname)
 			 "                               ALTER OWNER commands to set ownership\n"));
 
 	printf(_("\nConnection options:\n"));
+	printf(_("  -d, --dbname=DBNAME      database name to connect to\n"));
 	printf(_("  -h, --host=HOSTNAME      database server host or socket directory\n"));
 	printf(_("  -p, --port=PORT          database server port number\n"));
 	printf(_("  -U, --username=NAME      connect as specified database user\n"));
-- 
1.7.10.4

0003-Add-d-option-to-pg_dumpall-for-specifying-a-connecti.patchtext/x-diff; name=0003-Add-d-option-to-pg_dumpall-for-specifying-a-connecti.patchDownload
>From 2586b7e0edbed46ba7fe97809e19fc53ac815e19 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 20 Feb 2013 20:56:17 +0200
Subject: [PATCH 3/3] Add -d option to pg_dumpall, for specifying a connection
 string.

Like with pg_basebackup and pg_receivexlog, it's a bit strange to call the
option -d/--dbname, when in fact you cannot pass a database name in it
(XXX: would it make sense to allow passing the "initial" database in it,
instead of -l). But that's what the option to pass a connection string is
called in other client tools, so seems good to be consistent.
---
 doc/src/sgml/ref/pg_dumpall.sgml |   12 +++
 src/bin/pg_dump/pg_dumpall.c     |  194 +++++++++++++++++++++++++++++---------
 2 files changed, 164 insertions(+), 42 deletions(-)

diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 253ee01..e57d597 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -406,6 +406,18 @@ PostgreSQL documentation
 
    <variablelist>
      <varlistentry>
+      <term><option>-d <replaceable class="parameter">connstr</replaceable></option></term>
+      <term><option>--dbname=<replaceable class="parameter">connstr</replaceable></option></term>
+      <listitem>
+       <para>
+        Specifies connection string options, used for connecting to server.
+        These options can be used along with other user supplied options.
+        In case of conflicting options, the user supplied option is used.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
       <term><option>-h <replaceable>host</replaceable></option></term>
       <term><option>--host=<replaceable>host</replaceable></option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index a6e6a0f..84280f8 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -56,13 +56,14 @@ static int	runPgDump(const char *dbname);
 static void buildShSecLabels(PGconn *conn, const char *catalog_name,
 				 uint32 objectId, PQExpBuffer buffer,
 				 const char *target, const char *objname);
-static PGconn *connectDatabase(const char *dbname, const char *pghost, const char *pgport,
+static PGconn *connectDatabase(const char *dbname, const char *connstr, const char *pghost, const char *pgport,
 	  const char *pguser, enum trivalue prompt_password, bool fail_on_error);
 static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 
 static char pg_dump_bin[MAXPGPATH];
 static PQExpBuffer pgdumpopts;
+static char *connstr = "";
 static bool skip_acls = false;
 static bool verbose = false;
 
@@ -91,6 +92,7 @@ main(int argc, char *argv[])
 		{"globals-only", no_argument, NULL, 'g'},
 		{"host", required_argument, NULL, 'h'},
 		{"ignore-version", no_argument, NULL, 'i'},
+		{"dbname", required_argument, NULL, 'd'},
 		{"database", required_argument, NULL, 'l'},
 		{"oids", no_argument, NULL, 'o'},
 		{"no-owner", no_argument, NULL, 'O'},
@@ -188,7 +190,7 @@ main(int argc, char *argv[])
 
 	pgdumpopts = createPQExpBuffer();
 
-	while ((c = getopt_long(argc, argv, "acf:gh:il:oOp:rsS:tU:vwWx", long_options, &optindex)) != -1)
+	while ((c = getopt_long(argc, argv, "acd:f:gh:i:l:oOp:rsS:tU:vwWx", long_options, &optindex)) != -1)
 	{
 		switch (c)
 		{
@@ -201,6 +203,10 @@ main(int argc, char *argv[])
 				output_clean = true;
 				break;
 
+			case 'd':
+				connstr = pg_strdup(optarg);
+				break;
+
 			case 'f':
 				filename = pg_strdup(optarg);
 				appendPQExpBuffer(pgdumpopts, " -f ");
@@ -213,8 +219,6 @@ main(int argc, char *argv[])
 
 			case 'h':
 				pghost = pg_strdup(optarg);
-				appendPQExpBuffer(pgdumpopts, " -h ");
-				doShellQuoting(pgdumpopts, pghost);
 				break;
 
 			case 'i':
@@ -235,8 +239,6 @@ main(int argc, char *argv[])
 
 			case 'p':
 				pgport = pg_strdup(optarg);
-				appendPQExpBuffer(pgdumpopts, " -p ");
-				doShellQuoting(pgdumpopts, pgport);
 				break;
 
 			case 'r':
@@ -258,8 +260,6 @@ main(int argc, char *argv[])
 
 			case 'U':
 				pguser = pg_strdup(optarg);
-				appendPQExpBuffer(pgdumpopts, " -U ");
-				doShellQuoting(pgdumpopts, pguser);
 				break;
 
 			case 'v':
@@ -370,7 +370,7 @@ main(int argc, char *argv[])
 	 */
 	if (pgdb)
 	{
-		conn = connectDatabase(pgdb, pghost, pgport, pguser,
+		conn = connectDatabase(pgdb, connstr, pghost, pgport, pguser,
 							   prompt_password, false);
 
 		if (!conn)
@@ -382,10 +382,10 @@ main(int argc, char *argv[])
 	}
 	else
 	{
-		conn = connectDatabase("postgres", pghost, pgport, pguser,
+		conn = connectDatabase("postgres", connstr, pghost, pgport, pguser,
 							   prompt_password, false);
 		if (!conn)
-			conn = connectDatabase("template1", pghost, pgport, pguser,
+			conn = connectDatabase("template1", connstr, pghost, pgport, pguser,
 								   prompt_password, true);
 
 		if (!conn)
@@ -400,6 +400,48 @@ main(int argc, char *argv[])
 	}
 
 	/*
+	 * Now that we're connected, build the stem of the connection string that
+	 * we're going to pass to pg_dump.
+	 */
+	{
+		PQconninfoOption *conn_opts;
+		PQconninfoOption *conn_opt;
+		PQExpBuffer buf;
+
+		/* Extract the effective options used for the connection */
+		conn_opts = PQconninfo(conn);
+		if (!conn_opts)
+		{
+			fprintf(stderr, _("%s: out of memory\n"), progname);
+			exit_nicely(1);
+		}
+
+		/*
+		 * Construct a new connection string in key/value format, from the
+		 * options used in the current connection. 'dbname' option is left
+		 * out, because that varies in each pg_dump invocation.
+		 */
+		buf = createPQExpBuffer();
+		for (conn_opt = conn_opts; conn_opt->keyword; conn_opt++)
+		{
+			if (conn_opt->val == NULL || *conn_opt->val == '\0')
+				continue;
+
+			if (strcmp(conn_opt->keyword, "dbname") == 0)
+				continue;
+
+			appendPQExpBuffer(buf, "%s=", conn_opt->keyword);
+			doConnStrQuoting(buf, conn_opt->val);
+			appendPQExpBuffer(buf, " ");
+		}
+		connstr = pg_strdup(buf->data);
+		destroyPQExpBuffer(buf);
+
+		fprintf(stderr, "connstr: %s\n", connstr);
+
+	}
+
+	/*
 	 * Open the output file if required, otherwise use stdout
 	 */
 	if (filename)
@@ -568,6 +610,7 @@ help(void)
 			 "                               ALTER OWNER commands to set ownership\n"));
 
 	printf(_("\nConnection options:\n"));
+	printf(_("  -d, --dbname=CONNSTR     connect to server using connection string\n"));
 	printf(_("  -h, --host=HOSTNAME      database server host or socket directory\n"));
 	printf(_("  -l, --database=DBNAME    alternative default database\n"));
 	printf(_("  -p, --port=PORT          database server port number\n"));
@@ -1630,7 +1673,7 @@ dumpDatabases(PGconn *conn)
 static int
 runPgDump(const char *dbname)
 {
-	PQExpBuffer connstr = createPQExpBuffer();
+	PQExpBuffer connstrbuf = createPQExpBuffer();
 	PQExpBuffer cmd = createPQExpBuffer();
 	int			ret;
 
@@ -1652,11 +1695,10 @@ runPgDump(const char *dbname)
 	 * database name as is, but if it contains any = characters, it would
 	 * incorrectly treat it as a connection string.
 	 */
-	appendPQExpBuffer(connstr, "dbname='");
-	doConnStrQuoting(connstr, dbname);
-	appendPQExpBuffer(connstr, "'");
+	appendPQExpBuffer(connstrbuf, "%sdbname=", connstr);
+	doConnStrQuoting(connstrbuf, dbname);
 
-	doShellQuoting(cmd, connstr->data);
+	doShellQuoting(cmd, connstrbuf->data);
 
 	appendPQExpBuffer(cmd, "%s", SYSTEMQUOTE);
 
@@ -1669,7 +1711,7 @@ runPgDump(const char *dbname)
 	ret = system(cmd->data);
 
 	destroyPQExpBuffer(cmd);
-	destroyPQExpBuffer(connstr);
+	destroyPQExpBuffer(connstrbuf);
 
 	return ret;
 }
@@ -1705,7 +1747,7 @@ buildShSecLabels(PGconn *conn, const char *catalog_name, uint32 objectId,
  * on failure, but preserve any prompted password for the next try.
  */
 static PGconn *
-connectDatabase(const char *dbname, const char *pghost, const char *pgport,
+connectDatabase(const char *dbname, const char *connection_string, const char *pghost, const char *pgport,
 	   const char *pguser, enum trivalue prompt_password, bool fail_on_error)
 {
 	PGconn	   *conn;
@@ -1723,30 +1765,77 @@ connectDatabase(const char *dbname, const char *pghost, const char *pgport,
 	 */
 	do
 	{
-#define PARAMS_ARRAY_SIZE	7
-		const char **keywords = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*keywords));
-		const char **values = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*values));
-
-		keywords[0] = "host";
-		values[0] = pghost;
-		keywords[1] = "port";
-		values[1] = pgport;
-		keywords[2] = "user";
-		values[2] = pguser;
-		keywords[3] = "password";
-		values[3] = password;
-		keywords[4] = "dbname";
-		values[4] = dbname;
-		keywords[5] = "fallback_application_name";
-		values[5] = progname;
-		keywords[6] = NULL;
-		values[6] = NULL;
+		int			argcount = 6;
+		const char **keywords;
+		const char **values;
+		PQconninfoOption *conn_opts = NULL;
+		PQconninfoOption *conn_opt;
+		char	   *err_msg = NULL;
+		int			i = 0;
+
+		/*
+		 * Merge the connection info inputs given in form of connection string
+		 * and other options.
+		 */
+		if (connection_string)
+		{
+			conn_opts = PQconninfoParse(connection_string, &err_msg);
+			if (conn_opts == NULL)
+			{
+				fprintf(stderr, "%s: %s\n", progname, err_msg);
+				return NULL;
+			}
+
+			for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
+			{
+				if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
+					argcount++;
+			}
+
+			keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
+			values = pg_malloc0((argcount + 1) * sizeof(*values));
+
+			for (conn_opt = conn_opts; conn_opt->keyword != NULL; conn_opt++)
+			{
+				if (conn_opt->val != NULL && conn_opt->val[0] != '\0')
+				{
+					keywords[i] = conn_opt->keyword;
+					values[i] = conn_opt->val;
+					i++;
+				}
+			}
+		}
+		else
+		{
+			keywords = pg_malloc0((argcount + 1) * sizeof(*keywords));
+			values = pg_malloc0((argcount + 1) * sizeof(*values));
+		}
+
+		keywords[i] = "host";
+		values[i] = pghost;
+		i++;
+		keywords[i] = "port";
+		values[i] = pgport;
+		i++;
+		keywords[i] = "user";
+		values[i] = pguser;
+		i++;
+		keywords[i] = "password";
+		values[i] = password;
+		i++;
+		keywords[i] = "dbname";
+		values[i] = dbname;
+		i++;
+		keywords[i] = "fallback_application_name";
+		values[i] = progname;
+		i++;
 
 		new_pass = false;
 		conn = PQconnectdbParams(keywords, values, true);
 
 		free(keywords);
 		free(values);
+		PQconninfoFree(conn_opts);
 
 		if (!conn)
 		{
@@ -1917,15 +2006,36 @@ dumpTimestamp(char *msg)
 static void
 doConnStrQuoting(PQExpBuffer buf, const char *str)
 {
-	while (*str)
+	const char *s;
+	bool needquotes;
+
+	needquotes = false;
+	for (s = str; *s; s++)
+	{
+		if (!((*s >= 'a' && *s <= 'z') || (*s >= 'A' && *s <= 'Z') ||
+			  (*s >= '0' && *s <= '9') || *s == '_' || *s == '.'))
+		{
+			needquotes = true;
+			break;
+		}
+	}
+
+	if (needquotes)
 	{
-		/* ' and \ must be escaped by to \' and \\ */
-		if (*str == '\'' || *str == '\\')
-			appendPQExpBufferChar(buf, '\\');
+		appendPQExpBufferChar(buf, '\'');
+		while (*str)
+		{
+			/* ' and \ must be escaped by to \' and \\ */
+			if (*str == '\'' || *str == '\\')
+				appendPQExpBufferChar(buf, '\\');
 
-		appendPQExpBufferChar(buf, *str);
-		str++;
+			appendPQExpBufferChar(buf, *str);
+			str++;
+		}
+		appendPQExpBufferChar(buf, '\'');
 	}
+	else
+		appendPQExpBufferStr(buf, str);
 }
 
 /*
-- 
1.7.10.4

#20Phil Sorber
phil@omniti.com
In reply to: Heikki Linnakangas (#19)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Wed, Feb 20, 2013 at 2:16 PM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:

Where does this leave the PQconninfoParseParams/PQconninfodefaultsMerge
patch? I'm not sure. Somehow I thought it would be necessary for this work,
but it wasn't. I didn't remember that we already have PQconninfoParse()
function, which was enough. So, what's the use case for those functions?

I don't think that there is an immediate case. I still think they are
useful, and would be more useful if we had some other functions that
took PQconninfoOption. But the original reason for their being has
been circumvented and I think we should just push them off to next
release commit fest and discuss them then.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Amit Kapila
amit.kapila@huawei.com
In reply to: Heikki Linnakangas (#19)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Thursday, February 21, 2013 12:46 AM Heikki Linnakangas wrote:

On 20.02.2013 11:42, Amit Kapila wrote:

The patch for providing connection string for pg_basebackup,
pg_receivexlog, pg_dump and pg_dumpall is attached with this mail.

Thanks. Now that I look at this patch, I realize that we don't actually
need these new functions for pg_basebackup and friends after all. We
already have PQconninfoParse(), we can just use that.

pg_dump can already take a connection string:

pg_dump "dbname=postgres port=5432"

For consistency with psql and other tools, perhaps we should add a "-d"
option to pg_dump, so that you could do:

pg_dump -d "dbname=postgres port=5432"

It'd be nice to call the option -d or --dbname in all the tools. That's
a bit confusing for pg_basebackup and pg_receivexlog, as it can *not*
actually be a database name, but it would be otherwise consistent with
the other commands.

I came up with the attached three patches. The first adds -d/--dbname
option to pg_basebackup and pg_receivexlog.

Patch-1 is working fine.

The second adds it to
pg_dump, per above. The third adds it to pg_dumpall.

The third patch is a bit complicated. It first parses the user-
specified connection string using PQconninfoParse, so that it can merge
in some extra keywords: user, host, password, dbname and
fallback_application_name. It then calls PQconnectdbParams with the
keyword/value pairs. After making the initial connection to postgres or
template1 database, it calls PQconninfo() to again extract the
keyword/value pairs in effect in the connection, and constructs a new
connection string from them. That new connection string is then passed
to pg_dump on the command line, with the database name appended to it.

That seems to work, although it's perhaps a bit Rube Goldbergian. One
step of deparsing and parsing could be avoided by keeping the
keyword/value pairs from the first PQconninfoParse() call, instead of
constructing them again with PQconninfo(). I'll experiment with that
tomorrow.

I think this whole new logic in pg_dumpall is needed because in
pg_dump(Patch-2),
we have used dbname for new option.
In pg_dump, if for new option (-d) we use new variable conn_str similar as
we used at other places and change
the ConnectDatabase() in file pg_backup_db.c similar to GetConnection of
pg_basebackup, we might not
need the new logic of deparsing and parsing in pg_dumpall.

Am I missing something or do you feel this will be more cumbersome than
current Patch-2 & Patch-3?

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#22Heikki Linnakangas
hlinnakangas@vmware.com
In reply to: Amit Kapila (#21)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On 21.02.2013 16:09, Amit Kapila wrote:

On Thursday, February 21, 2013 12:46 AM Heikki Linnakangas wrote:

The second adds it to
pg_dump, per above. The third adds it to pg_dumpall.

The third patch is a bit complicated. It first parses the user-
specified connection string using PQconninfoParse, so that it can merge
in some extra keywords: user, host, password, dbname and
fallback_application_name. It then calls PQconnectdbParams with the
keyword/value pairs. After making the initial connection to postgres or
template1 database, it calls PQconninfo() to again extract the
keyword/value pairs in effect in the connection, and constructs a new
connection string from them. That new connection string is then passed
to pg_dump on the command line, with the database name appended to it.

That seems to work, although it's perhaps a bit Rube Goldbergian. One
step of deparsing and parsing could be avoided by keeping the
keyword/value pairs from the first PQconninfoParse() call, instead of
constructing them again with PQconninfo(). I'll experiment with that
tomorrow.

I did the above, keeping the keyword/value pairs from the
PQconninfoParse() step instead.

I think this whole new logic in pg_dumpall is needed because in
pg_dump(Patch-2),
we have used dbname for new option.
In pg_dump, if for new option (-d) we use new variable conn_str similar as
we used at other places and change
the ConnectDatabase() in file pg_backup_db.c similar to GetConnection of
pg_basebackup, we might not
need the new logic of deparsing and parsing in pg_dumpall.

Yeah, could be. However, it's good to call the option -d for the sake of
consistency. Also, there would be some confusion if there were two ways
to pass a connection string (we'd stil have to support passing a
connection string as the database name, for backwards-compatibility).

I've committed those patches, with some further changes. If you have the
time, please take another look at the committed patches. Thanks!

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#23Amit Kapila
amit.kapila@huawei.com
In reply to: Heikki Linnakangas (#22)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Monday, February 25, 2013 11:26 PM Heikki Linnakangas wrote:

On 21.02.2013 16:09, Amit Kapila wrote:

On Thursday, February 21, 2013 12:46 AM Heikki Linnakangas wrote:

I've committed those patches, with some further changes. If you have
the
time, please take another look at the committed patches. Thanks!

1. For pg_dumpall, incase user gives dbname in connection string, for ex.
"user=amit dbname=db_regress port=5434"
it will be ignored and postgres (default database name) will be used.

2. For pg_basebackup, no need to check if(conn_opts) before PQconninfoFree
as it will internally take care of
NULL. This will not cause any harm, just an observation.

With Regards,
Amit Kapila

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#24Heikki Linnakangas
hlinnakangas@vmware.com
In reply to: Amit Kapila (#23)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On 26.02.2013 09:06, Amit Kapila wrote:

On Monday, February 25, 2013 11:26 PM Heikki Linnakangas wrote:

On 21.02.2013 16:09, Amit Kapila wrote:

On Thursday, February 21, 2013 12:46 AM Heikki Linnakangas wrote:

I've committed those patches, with some further changes. If you have
the
time, please take another look at the committed patches. Thanks!

1. For pg_dumpall, incase user gives dbname in connection string, for ex.
"user=amit dbname=db_regress port=5434"
it will be ignored and postgres (default database name) will be used.

Yeah, that is mentioned in the docs. The other option would be to use
the database name from the connection string for the initial connection,
ie. the same as specifying a database name with the -l option. Yet
another option is to throw an error if database name was given. If we
make it an error, then we should make it an error in pg_basebackup and
pg_receivexlog too. I'm all ears for opinions on this.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#25Amit Kapila
amit.kapila@huawei.com
In reply to: Heikki Linnakangas (#24)
Re: [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

On Tuesday, February 26, 2013 1:36 PM Heikki Linnakangas wrote:

On 26.02.2013 09:06, Amit Kapila wrote:

On Monday, February 25, 2013 11:26 PM Heikki Linnakangas wrote:

On 21.02.2013 16:09, Amit Kapila wrote:

On Thursday, February 21, 2013 12:46 AM Heikki Linnakangas wrote:

I've committed those patches, with some further changes. If you have
the
time, please take another look at the committed patches. Thanks!

1. For pg_dumpall, incase user gives dbname in connection string, for

ex.

"user=amit dbname=db_regress port=5434"
it will be ignored and postgres (default database name) will be

used.

Yeah, that is mentioned in the docs.

Sorry, I have missed that part, this just came to me when I was looking at
the code.

The other option would be to use
the database name from the connection string for the initial
connection,
ie. the same as specifying a database name with the -l option.

This is what come to my mind initially, as if it is allowed with -l option
it is better if the same should be allowed through connection string as
well.
We are ignoring for pg_basebackup as well, but in pg_basebackup there is no
way for user to
specify database name.

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers