PATCH: psql tab completion for SELECT

Started by Edmund Hornerabout 8 years ago47 messages
#1Edmund Horner
ejrh00@gmail.com
1 attachment(s)

Hi pgsql-hackers,

Here's a little draft patch to add *some* tab completion ability for
SELECT in psql. I have often missed the ability, especially with
invocations of utility functions.

It would be nice to be able to suggest column names from the relevant
tables in the query, but, as the SQL language puts the column list
before the FROM clause, we have to use columns from all tables; I'm
certainly not the first to be frustrated by this language feature, but
we can't do much about it.

What my patch does:

For a command line with the single word SELECT, column names and
function names will be used for completions of the next word.
Function names have a "(" suffix added, largely because it makes them
easier to distinguish in the completion list.

Only the first select-list item can be tab-completed; i.e. SELECT foo,
b<tab> won't find column bar.

Examples:

postgres=# select rel<tab>
relacl relchecks relhasindex
relhassubclass (etc.)

postgres=# select str<tab>
string_to_array( strip( strpos(

How it works:

The implementation uses a normal non-schema query, because columns
don't have schemas.

The eligible columns are normal, visible columns.

I have tried to filter out the various functions which aren't likely
to be directly used in queries.

The eligible functions are those which are:
- visible
- not aggregate or window functions
- not RI_FKey_* functions
- not support functions for types, aggregates, operators, languages,
casts, access methods.

Completions for FROM, WHERE, etc. still work, since the additional
completions are only used immediately after the single word SELECT.

Is this likely to be a useful addition to PostgreSQL?

If so, I'll try to get the patch to a good standard. I am not aiming
for complete support for the SELECT grammar, but just the low-hanging
fruit.

I'm not aware of existing tests for psql tab completion. But I ran
"make check" anyway, with no problems.

Some other questions about how it should be done:

- Are my criteria for the columns and function names appropriate?

- Should I try to use a SchemaQuery so that function schema names can be used?

- Should I try to support simple cases of multiple columns? (How?
We can use TailMatches1(",") but how do we tell we aren't into the
FROM-clause or later?)

- How do we make it work with older servers, e.g. those that predate
some of the newer columns used in the function criteria?

Cheers,
Edmund Horner

Attachments:

psql-select-tab-completion-v1.patchapplication/octet-stream; name=psql-select-tab-completion-v1.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a09c49d..6e069c9 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -982,6 +982,28 @@ static const SchemaQuery Query_for_list_of_statistics = {
 "       and pg_catalog.pg_table_is_visible(c2.oid)"\
 "       and c2.relispartition = 'true'"
 
+#define Query_for_list_of_selectable_functions_or_attributes \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND NOT proisagg AND NOT proiswindow AND proname NOT LIKE 'RI_FKey_%%' "\
+"      AND oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "\
+"      AND oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn,aggcombinefn,aggserialfn,aggdeserialfn,aggmtransfn,aggminvtransfn,aggmfinalfn]) FROM pg_aggregate) "\
+"      AND oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "\
+"      AND oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "\
+"      AND oid NOT IN (SELECT castfunc FROM pg_cast) "\
+"      AND oid NOT IN (SELECT protransform FROM pg_proc) "\
+"      AND oid NOT IN (SELECT amhandler FROM pg_am) "\
+"      AND oid NOT IN (SELECT amproc FROM pg_amproc) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
 /*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
@@ -3172,7 +3194,10 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (Matches1("SELECT"))
+	{
+		COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes);
+	}
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
#2David Fetter
david@fetter.org
In reply to: Edmund Horner (#1)
Re: PATCH: psql tab completion for SELECT

On Mon, Nov 06, 2017 at 05:28:55PM +1300, Edmund Horner wrote:

Hi pgsql-hackers,

Here's a little draft patch to add *some* tab completion ability for
SELECT in psql. I have often missed the ability, especially with
invocations of utility functions.

It would be nice to be able to suggest column names from the
relevant tables in the query, but, as the SQL language puts the
column list before the FROM clause, we have to use columns from all
tables; I'm certainly not the first to be frustrated by this
language feature, but we can't do much about it.

What my patch does:

For a command line with the single word SELECT, column names and
function names will be used for completions of the next word.
Function names have a "(" suffix added, largely because it makes
them easier to distinguish in the completion list.

Only the first select-list item can be tab-completed; i.e. SELECT
foo, b<tab> won't find column bar.

That can be fixed later.

Examples:

postgres=# select rel<tab>
relacl relchecks relhasindex
relhassubclass (etc.)

postgres=# select str<tab>
string_to_array( strip( strpos(

Neat!

Please add this to the upcoming (2018-01) commitfest at
https://commitfest.postgresql.org/

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Michael Paquier
michael.paquier@gmail.com
In reply to: David Fetter (#2)
Re: PATCH: psql tab completion for SELECT

On Mon, Nov 13, 2017 at 5:13 AM, David Fetter <david@fetter.org> wrote:

Please add this to the upcoming (2018-01) commitfest at
https://commitfest.postgresql.org/

You may want to scan the following thread as well:
/messages/by-id/1328820579.11241.4.camel@vanquo.pezone.net
And also you should be careful about things like WITH clauses...
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#1)
Re: [HACKERS] PATCH: psql tab completion for SELECT

On 11/06/2017 05:28 AM, Edmund Horner wrote:

Hi pgsql-hackers,

Here's a little draft patch to add *some* tab completion ability for
SELECT in psql. I have often missed the ability, especially with
invocations of utility functions.

Yes, indeed! The vast majority of what I use interactive psql for is to
call functions. The fact that it does columns as well is nice, and not
doing anything after the first isn't a problem as we have many other
places where that's the case. It's annoying, but consistent with
current annoying behavior. So no points off for that.

What my patch does:

For a command line with the single word SELECT, column names and
function names will be used for completions of the next word.
Function names have a "(" suffix added, largely because it makes them
easier to distinguish in the completion list.

And because the user is going to have to type one anyway. The
completion mechanism adds a space after the paren, which is a needle in
the eye of my OCD, but that's not this patch's fault.

Only the first select-list item can be tab-completed; i.e. SELECT foo,
b<tab> won't find column bar.

That's not a problem for me.

How it works:

The implementation uses a normal non-schema query, because columns
don't have schemas.

The eligible columns are normal, visible columns.

At first I thought this was too broad, but of course it's not because
the column will be visible if the user schema-qualifies the table it's in.

I have tried to filter out the various functions which aren't likely
to be directly used in queries.

The eligible functions are those which are:
- visible

Yes.

- not aggregate or window functions

Why? I think this is a big mistake.

- not RI_FKey_* functions

Agreed.

- not support functions for types, aggregates, operators, languages,
casts, access methods.

That list looks good to me. But you forgot to exclude trigger functions.

Is this likely to be a useful addition to PostgreSQL?

Yes, very much so.

Some other questions about how it should be done:

- Are my criteria for the columns and function names appropriate?

No, I don't think so, as mentioned above.

- Should I try to use a SchemaQuery so that function schema names can be used?

- Should I try to support simple cases of multiple columns? (How?
We can use TailMatches1(",") but how do we tell we aren't into the
FROM-clause or later?)

I think these two can be pushed to a later patch. I'd rather have what
you've got now than nothing at all.

- How do we make it work with older servers, e.g. those that predate
some of the newer columns used in the function criteria?

This is something that's going to have to be addressed if this patch is
to be committed. The way to do this is by writing a query for each
different version you need to support and then call the right one
depending on the pset.sversion global variable.

Marking as Waiting on Author.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#5Edmund Horner
ejrh00@gmail.com
In reply to: Vik Fearing (#4)
Re: [HACKERS] PATCH: psql tab completion for SELECT

Hi Vik, thanks so much for the comments and the offer to review!

I kept a low profile after my first message as there was already a
commitfest in progress, but now I'm working on a V2 patch.

I will include aggregate and window functions as you suggest. And
I'll exclude trigger functions.

I'd like to exclude more if we could, because there are already over
1000 completions on an empty database. I had thought of filtering out
functions with an argument of type internal but couldn't figure out
how to interrogate the proargtypes oidvector in a nice way.

Regarding support for older versions, psql fails silently if a tab
completion query fails. We could just let it do this, which is what
happens with, for example, ALTER PUBLICATION against a 9.6 server. I
can't see any existing completions that check the server version --
but completions that don't work against older versions, like ALTER
PUBLICATION, also aren't useful for older versions. SELECT is a bit
different as it can be useful against older versions that don't have
the pg_aggregate.aggcombinefn that my query uses for filtering out
aggregation support functions.

There's also the small irritation that when a completion query fails,
it aborts the user's current transaction to provide an incentive for
handling older versions gracefully.

Regarding multiple columns, I have an idea that if we check that:

a) the last of any SELECT/WHERE/GROUP BY/etc.-level keyword is a
SELECT (i.e. that we're in the SELECT clause of the query), and
b) the last word was a comma (or ended in a comma).
we can then proffer the column/function completions.

There may be other completions that could benefit from such a check,
e.g. table names after a comma in the FROM clause, but I just want to
get SELECT working first.

#6Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#5)
Re: [HACKERS] PATCH: psql tab completion for SELECT

On 01/10/2018 06:38 AM, Edmund Horner wrote:

Hi Vik, thanks so much for the comments and the offer to review!

I kept a low profile after my first message as there was already a
commitfest in progress, but now I'm working on a V2 patch.

I will include aggregate and window functions as you suggest. And
I'll exclude trigger functions.

Thanks.

I'd like to exclude more if we could, because there are already over
1000 completions on an empty database. I had thought of filtering out
functions with an argument of type internal but couldn't figure out
how to interrogate the proargtypes oidvector in a nice way.

If you just want to do internal,

WHERE regtype 'internal' <> ALL (proargtypes)

but if you'll want to add more banned types, then

WHERE NOT (proargtypes::regtype[] && array['internal',
'unknown']::regtype[])

This is a very good test to add.

Regarding support for older versions, psql fails silently if a tab
completion query fails.

No it doesn't, see below.

We could just let it do this, which is what
happens with, for example, ALTER PUBLICATION against a 9.6 server. I
can't see any existing completions that check the server version --
but completions that don't work against older versions, like ALTER
PUBLICATION, also aren't useful for older versions. SELECT is a bit
different as it can be useful against older versions that don't have
the pg_aggregate.aggcombinefn that my query uses for filtering out
aggregation support functions.

That's a bug in my opinion, but not one that needs to be addressed by
this patch.

There are no completions that cater to the server version (yet) but all
the \d stuff certainly does. You can see that in action in
src/bin/psql/describe.c as well as all over the place in pg_dump and the
like.

There's also the small irritation that when a completion query fails,
it aborts the user's current transaction to provide an incentive for
handling older versions gracefully.

That is actively hostile and not at all what I would consider "silently
failing". If you don't want to do the multiple versions thing, you
should at least check if you're on 9.6+ before issuing the query.

Regarding multiple columns, I have an idea that if we check that:

a) the last of any SELECT/WHERE/GROUP BY/etc.-level keyword is a
SELECT (i.e. that we're in the SELECT clause of the query), and
b) the last word was a comma (or ended in a comma).
we can then proffer the column/function completions.

There may be other completions that could benefit from such a check,
e.g. table names after a comma in the FROM clause, but I just want to
get SELECT working first.

I would like to see this as a separate patch. Let's keep this one
simple and just do like the other things currently do.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#7Edmund Horner
ejrh00@gmail.com
In reply to: Vik Fearing (#6)
Re: [HACKERS] PATCH: psql tab completion for SELECT

On 11 January 2018 at 03:28, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

On 01/10/2018 06:38 AM, Edmund Horner wrote:

Regarding support for older versions, psql fails silently if a tab
completion query fails.

No it doesn't, see below.

We could just let it do this, which is what
happens with, for example, ALTER PUBLICATION against a 9.6 server. I
can't see any existing completions that check the server version --
but completions that don't work against older versions, like ALTER
PUBLICATION, also aren't useful for older versions. SELECT is a bit
different as it can be useful against older versions that don't have
the pg_aggregate.aggcombinefn that my query uses for filtering out
aggregation support functions.

That's a bug in my opinion, but not one that needs to be addressed by
this patch.

There are no completions that cater to the server version (yet) but all
the \d stuff certainly does. You can see that in action in
src/bin/psql/describe.c as well as all over the place in pg_dump and the
like.

There's also the small irritation that when a completion query fails,
it aborts the user's current transaction to provide an incentive for
handling older versions gracefully.

That is actively hostile and not at all what I would consider "silently
failing". If you don't want to do the multiple versions thing, you
should at least check if you're on 9.6+ before issuing the query.

There's also the less-critical problem that you can't complete
anything from an already-failed transaction.

Let's just talk about a separate patch that might improve this.

I can think of two options:

1. Use a separate connection for completions. The big problem with
this is people want to complete on objects created in their current
transaction. Maybe there's a way to use SET TRANSACTION SNAPSHOT to
access the user's transaction but this seems far too intrusive just
for a bit of tab completion.

2. Use savepoints. In exec_query you'd have:

SAVEPOINT _psql_tab_completion;
run the query
ROLLBACK TO _psql_tab_completion; // Just in case there was an
error, but safe to do anyway.
RELEASE SAVEPOINT _psql_tab_completion; // This may not be worth doing.

It doesn't help with tab completion in already-failed transactions.
But would it be a reasonable way to make tab completion safer? I
don't know whether savepoint creation/rollback/release has some
cumulative cost that we'd want to avoid incurring.

#8Edmund Horner
ejrh00@gmail.com
In reply to: Edmund Horner (#1)
1 attachment(s)
Re: PATCH: psql tab completion for SELECT

Hi, here's a new patch.

This one makes some changes to the criteria for which functions to
include; namely filtering out trigger functions and those that take or
return values of type "internal"; and including aggregate and window
functions. Some of the other checks could be removed as they were
covered by the "internal" check.

It also uses the server version to determine which query to run. I
have not written a custom query for every version from 7.1! I've
split up the server history into:

pre-7.3 - does not even have pg_function_is_visible. Not supported.
pre-9.0 - does not have several support functions in types, languages,
etc. We don't bother filtering using columns in other tables.
pre-9.6 - does not have various aggregate support functions.
9.6 or later - the full query

I was able to test against 9.2, 9.6, and 10 servers, but compiling and
testing the older releases was a bit much for a Friday evening. I'm
not sure there's much value in support for old servers, as long as we
can make completion queries fail a bit more gracefully.

Edmund

Attachments:

psql-select-tab-completion-v2.patchapplication/octet-stream; name=psql-select-tab-completion-v2.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b51098d..c1bf7fb 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1012,6 +1012,67 @@ static const SchemaQuery Query_for_list_of_statistics = {
 "       and pg_catalog.pg_table_is_visible(c2.oid)"\
 "       and c2.relispartition = 'true'"
 
+/* For server versions prior to 9.0, do not check tables referencing pg_proc.
+   This query should work for server versions 7.3 and later; prior to that,
+   even pg_function_is_visible doesn't exist. */
+#define Query_for_list_of_selectable_functions_or_attributes_PRE_90 \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"      AND 'internal'::regtype != ALL (proargtypes) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
+/* For server versions prior to 9.6, pg_aggregate lacks several columns for support functions. */
+#define Query_for_list_of_selectable_functions_or_attributes_PRE_96 \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"      AND 'internal'::regtype != ALL (proargtypes) "\
+"      AND oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "\
+"      AND oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn]) FROM pg_aggregate) "\
+"      AND oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "\
+"      AND oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "\
+"      AND oid NOT IN (SELECT castfunc FROM pg_cast) "\
+"      AND oid NOT IN (SELECT amproc FROM pg_amproc) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
+#define Query_for_list_of_selectable_functions_or_attributes \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"      AND 'internal'::regtype != ALL (proargtypes) "\
+"      AND oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "\
+"      AND oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn,aggcombinefn,aggserialfn,aggdeserialfn,aggmtransfn,aggminvtransfn,aggmfinalfn]) FROM pg_aggregate) "\
+"      AND oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "\
+"      AND oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "\
+"      AND oid NOT IN (SELECT castfunc FROM pg_cast) "\
+"      AND oid NOT IN (SELECT amproc FROM pg_amproc) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
 /*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
@@ -3221,7 +3282,15 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (Matches1("SELECT") && pset.sversion >= 70300)
+	{
+		if (pset.sversion < 90000)
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes_PRE_90);
+		else if (pset.sversion < 90600)
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes_PRE_96);
+		else
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes);
+	}
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
#9Edmund Horner
ejrh00@gmail.com
In reply to: Edmund Horner (#1)
1 attachment(s)
Re: PATCH: psql tab completion for SELECT

And here's a patch to add savepoint protection for tab completion.

It could definitely use some scrutiny to make sure it's not messing up
the user's transaction.

I added some error checking for the savepoint creation and the
rollback, and then wrapped it in #ifdef NOT_USED (just as the query
error checking is) as I wasn't sure how useful it is for normal use.

But I do feel that if psql unexpectedly messes up the transaction, it
should report it somehow. How can we tell that we've messed it up for
the user?

Cheers,
Edmund

Attachments:

psql-tab-completion-savepoint-v1.patchapplication/octet-stream; name=psql-tab-completion-savepoint-v1.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index c1bf7fb..fc951ad 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4295,10 +4295,30 @@ static PGresult *
 exec_query(const char *query)
 {
        PGresult   *result;
+       bool rollback_to_tab_completion_savepoint = false;
 
        if (query == NULL || !pset.db || PQstatus(pset.db) != CONNECTION_OK)
                return NULL;
 
+       /* If the server supports it, create a savepoint before running the tab completion query
+        * in case it fails. */
+       if (pset.sversion >= 80000)
+       {
+               PGTransactionStatusType transaction_status = PQtransactionStatus(pset.db);
+
+               if (transaction_status == PQTRANS_INTRANS)
+               {
+                       PGresult *savepoint_result = PQexec(pset.db, "SAVEPOINT pg_psql_tab_completion_savepoint");
+                       if (PQresultStatus(savepoint_result) == PGRES_COMMAND_OK)
+                               rollback_to_tab_completion_savepoint = true;
+#ifdef NOT_USED
+                       else
+                               psql_error("could not create savepoint for tab completion: %s", PQerrorMessage(pset.db));
+#endif
+                       PQclear(savepoint_result);
+               }
+       }
+
        result = PQexec(pset.db, query);
 
        if (PQresultStatus(result) != PGRES_TUPLES_OK)
@@ -4311,6 +4331,18 @@ exec_query(const char *query)
                result = NULL;
        }
 
+       /* If a savepoint was created above, roll back to it.  We do this regardless of whether the
+          above query was successful, because all tab completion queries should have no effect. */
+       if (rollback_to_tab_completion_savepoint)
+       {
+               PGresult *rollback_result = PQexec(pset.db, "ROLLBACK TO pg_psql_tab_completion_savepoint");
+#ifdef NOT_USED
+               if (PQresultStatus(rollback_result) != PGRES_COMMAND_OK)
+                       psql_error("could not roll back to tab completion savepoint: %s", PQerrorMessage(pset.db));
+#endif
+               PQclear(rollback_result);
+       }
+
        return result;
 }
#10Andres Freund
andres@anarazel.de
In reply to: Edmund Horner (#9)
Re: PATCH: psql tab completion for SELECT

On January 14, 2018 5:12:37 PM PST, Edmund Horner <ejrh00@gmail.com> wrote:

And here's a patch to add savepoint protection for tab completion.

It'd be good to explain what that means, so people don't have to read the patch to be able to discuss whether this is a good idea.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

#11Edmund Horner
ejrh00@gmail.com
In reply to: Andres Freund (#10)
Re: PATCH: psql tab completion for SELECT

On 15 January 2018 at 14:20, Andres Freund <andres@anarazel.de> wrote:

On January 14, 2018 5:12:37 PM PST, Edmund Horner <ejrh00@gmail.com> wrote:

And here's a patch to add savepoint protection for tab completion.

It'd be good to explain what that means, so people don't have to read the patch to be able to discuss whether this is a good idea.

Good idea.

In psql if you have readline support and press TAB, psql will often
run a DB query to get a list of possible completions to type on your
current command line.

It uses the current DB connection for this, which means that if the
tab completion query fails (e.g. because psql is querying catalog
objects that doesn't exist in your server), the current transaction
(if any) fails. An example of this happening is:

$ psql -h old_database_server
psql (10.1, server 9.2.24)
Type "help" for help.

postgres=# begin;
BEGIN
postgres=# create table foo (id int);
CREATE TABLE
postgres=# alter subscription <TAB>

(no tab completions because the pg_subscription table doesn't
exist on 9.2! User realises their mistake and types a different
command)

postgres=# select * from foo;
ERROR: current transaction is aborted, commands ignored until end
of transaction block

This patch:
- checks that the server supports savepoints (version 8.0 and later)
and that the user is currently idle in a transaction
- if so, creates a savepoint just before running a tab-completion query
- rolls back to that savepoint just after running the query

The result is that on an 8.0 or later server, the user's transaction
is still ok:

$ psql -h old_database_server
psql (11devel, server 9.2.24)
Type "help" for help.

postgres=# begin;
BEGIN
postgres=# create table foo (id int);
CREATE TABLE
postgres=# alter subscription <TAB>

(again, no tab completions)

postgres=# select * from p;
id
----
(0 rows)

postgres=# commit;
COMMIT

Note that only the automatic tab-completion query is protected; the
user can still fail their transaction by typing an invalid command
like ALTER SUBSCRIPTION ;.

#12Andres Freund
andres@anarazel.de
In reply to: Edmund Horner (#11)
Re: PATCH: psql tab completion for SELECT

On January 14, 2018 5:44:01 PM PST, Edmund Horner <ejrh00@gmail.com> wrote:

On 15 January 2018 at 14:20, Andres Freund <andres@anarazel.de> wrote:

On January 14, 2018 5:12:37 PM PST, Edmund Horner <ejrh00@gmail.com>

wrote:

And here's a patch to add savepoint protection for tab completion.

It'd be good to explain what that means, so people don't have to read

the patch to be able to discuss whether this is a good idea.

Good idea.

In psql if you have readline support and press TAB, psql will often
run a DB query to get a list of possible completions to type on your
current command line.

It uses the current DB connection for this, which means that if the
tab completion query fails (e.g. because psql is querying catalog
objects that doesn't exist in your server), the current transaction
(if any) fails. An example of this happening is:

Ah, that's what I thought. I don't think this is the right fix.

pg_subscription table doesn't
exist on 9.2! User realises their mistake and types a different
command)

postgres=# select * from foo;
ERROR: current transaction is aborted, commands ignored until end
of transaction block

All worries like this are supposed to check the server version.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

#13Edmund Horner
ejrh00@gmail.com
In reply to: Andres Freund (#12)
Re: PATCH: psql tab completion for SELECT

On 15 January 2018 at 15:45, Andres Freund <andres@anarazel.de> wrote:

On January 14, 2018 5:44:01 PM PST, Edmund Horner <ejrh00@gmail.com> wrote:

In psql if you have readline support and press TAB, psql will often
run a DB query to get a list of possible completions to type on your
current command line.

It uses the current DB connection for this, which means that if the
tab completion query fails (e.g. because psql is querying catalog
objects that doesn't exist in your server), the current transaction
(if any) fails. An example of this happening is:

Ah, that's what I thought. I don't think this is the right fix.

pg_subscription table doesn't
exist on 9.2! User realises their mistake and types a different
command)

postgres=# select * from foo;
ERROR: current transaction is aborted, commands ignored until end
of transaction block

All worries like this are supposed to check the server version.

In psql there are around 200 such tab completion queries, none of
which checks the server version. Many would cause the user's
transaction to abort if invoked on an older server. Identifying the
appropriate server versions for each one would be quite a bit of work.

Is there a better way to make this more robust?

#14Tom Lane
tgl@sss.pgh.pa.us
In reply to: Edmund Horner (#13)
Re: PATCH: psql tab completion for SELECT

Edmund Horner <ejrh00@gmail.com> writes:

On 15 January 2018 at 15:45, Andres Freund <andres@anarazel.de> wrote:

All worries like this are supposed to check the server version.

In psql there are around 200 such tab completion queries, none of
which checks the server version. Many would cause the user's
transaction to abort if invoked on an older server. Identifying the
appropriate server versions for each one would be quite a bit of work.

Is there a better way to make this more robust?

Maybe it'd be worth the effort to wrap tab completion queries in
SAVEPOINT/RELEASE SAVEPOINT if we're inside a user transaction
(which we could detect from libpq's state, I believe).

That seems like an independent patch, but it'd be a prerequisite
if you want to issue tab completion queries with version dependencies.

A bigger point here is: do you really want SELECT tab completion
to work only against the latest and greatest server version?
That would become an argument against committing the feature at all;
maybe not enough to tip the scales against it, but still a demerit.

regards, tom lane

#15Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Tom Lane (#14)
Re: PATCH: psql tab completion for SELECT

On 01/18/2018 01:07 AM, Tom Lane wrote:

Edmund Horner <ejrh00@gmail.com> writes:

On 15 January 2018 at 15:45, Andres Freund <andres@anarazel.de> wrote:

All worries like this are supposed to check the server version.

In psql there are around 200 such tab completion queries, none of
which checks the server version. Many would cause the user's
transaction to abort if invoked on an older server. Identifying the
appropriate server versions for each one would be quite a bit of work.

Is there a better way to make this more robust?

Maybe it'd be worth the effort to wrap tab completion queries in
SAVEPOINT/RELEASE SAVEPOINT if we're inside a user transaction
(which we could detect from libpq's state, I believe).

That seems like an independent patch, but it'd be a prerequisite
if you want to issue tab completion queries with version dependencies.

A bigger point here is: do you really want SELECT tab completion
to work only against the latest and greatest server version?
That would become an argument against committing the feature at all;
maybe not enough to tip the scales against it, but still a demerit.

I don't really want such a patch. I use new psql on old servers all the
time.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#16Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#8)
Re: PATCH: psql tab completion for SELECT

On 01/12/2018 06:59 AM, Edmund Horner wrote:

Hi, here's a new patch.

Thanks, and sorry for the delay. I have reviewed this patch, but
haven't had time to put my thoughts together in a coherent message yet.
I will be able to do that tomorrow.

Thank you for your patience.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#17Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#8)
Re: PATCH: psql tab completion for SELECT

On 01/12/2018 06:59 AM, Edmund Horner wrote:

Hi, here's a new patch.

This one makes some changes to the criteria for which functions to
include; namely filtering out trigger functions and those that take or
return values of type "internal"; and including aggregate and window
functions. Some of the other checks could be removed as they were
covered by the "internal" check.

This looks better. One minor quibble I have is your use of != (which
PostgreSQL accepts) instead of <> (the SQL standard). I'll let the
committer worry about that.

It also uses the server version to determine which query to run. I
have not written a custom query for every version from 7.1! I've
split up the server history into:

pre-7.3 - does not even have pg_function_is_visible. Not supported.
pre-9.0 - does not have several support functions in types, languages,
etc. We don't bother filtering using columns in other tables.
pre-9.6 - does not have various aggregate support functions.
9.6 or later - the full query

I'm not sure how overboard we need to go with this going backwards so
what you did is probably fine.

I was able to test against 9.2, 9.6, and 10 servers, but compiling and
testing the older releases was a bit much for a Friday evening. I'm
not sure there's much value in support for old servers, as long as we
can make completion queries fail a bit more gracefully.

pg_dump stops at 8.0, we can surely do the same.

Now for some other points:

Your use of Matches1 is wrong, you should use TailMatches1 instead.
SELECT is a fully reserved keyword, so just checking if it's the
previous token is sufficient. By using Matches1, you miss cases like
this: SELECT (SELECT <tab>

It also occurred to me that SELECT isn't actually a complete command (or
whatever you want to call it), it's an abbreviation for SELECT ALL. So
you should check for SELECT, SELECT ALL, and SELECT DISTINCT. (The FROM
tab completion is missing this trick but that's a different patch)
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#18Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#1)
Re: PATCH: psql tab completion for SELECT

On Tue, Jan 23, 2018 at 4:17 AM, Edmund Horner <ejrh00@gmail.com> wrote:

Hi Vik, thanks for the feedback!

Don't forget to include the list :-)
I'm quoting the entirety of the message---which I would normally trim---for
the archives.

On 23 January 2018 at 07:41, Vik Fearing <vik.fearing@2ndquadrant.com>
wrote:

This looks better. One minor quibble I have is your use of != (which
PostgreSQL accepts) instead of <> (the SQL standard). I'll let the
committer worry about that.

There are both forms in the existing queries, but if <> is more
standard, we should use that.

It also uses the server version to determine which query to run. I
have not written a custom query for every version from 7.1! I've
split up the server history into:

pre-7.3 - does not even have pg_function_is_visible. Not supported.
pre-9.0 - does not have several support functions in types, languages,
etc. We don't bother filtering using columns in other tables.
pre-9.6 - does not have various aggregate support functions.
9.6 or later - the full query

I'm not sure how overboard we need to go with this going backwards so
what you did is probably fine.

What I did might be considered overboard, too. :)

If this were my patch, I'd have one query for 8.0, and then queries for all
currently supported versions if needed. If 9.2 only gets 8.0-level
features, that's fine by me. That limits us to a maximum of six queries.
Just because we keep support for dead versions doesn't mean we have to
retroactively develop for them. Au contraire.

I was able to test against 9.2, 9.6, and 10 servers, but compiling and
testing the older releases was a bit much for a Friday evening. I'm
not sure there's much value in support for old servers, as long as we
can make completion queries fail a bit more gracefully.

pg_dump stops at 8.0, we can surely do the same.

Ok. I'll try to do a bit more testing against servers in that range.

Now for some other points:

Your use of Matches1 is wrong, you should use TailMatches1 instead.
SELECT is a fully reserved keyword, so just checking if it's the
previous token is sufficient. By using Matches1, you miss cases like
this: SELECT (SELECT <tab>

It also occurred to me that SELECT isn't actually a complete command (or
whatever you want to call it), it's an abbreviation for SELECT ALL. So
you should check for SELECT, SELECT ALL, and SELECT DISTINCT. (The FROM
tab completion is missing this trick but that's a different patch)

Right. TailMatches it is.

(I was thinking we could preprocess the input to parts extraneous to
the current query for tab completion purposes, such as:
- Input up to and including "(", to tab complete a sub-query.
- Input inside "()", to ignore complete subqueries that might contain
keywords
- Everything inside quotes.
etc.
Which would be a step to support things like comma-separated SELECT,
FROM, GROUP BY items. But that's all way too complicated for this
patch.)

I'll make another patch version, with these few differences.

Great!
--

Vik Fearing +33 6 46 75 15
36http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et
Support

#19Edmund Horner
ejrh00@gmail.com
In reply to: Vik Fearing (#18)
1 attachment(s)
Re: PATCH: psql tab completion for SELECT

On 23 January 2018 at 21:47, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

Don't forget to include the list :-)
I'm quoting the entirety of the message---which I would normally trim---for
the archives.

Thanks for spotting that. Sorry list!

If this were my patch, I'd have one query for 8.0, and then queries for all
currently supported versions if needed. If 9.2 only gets 8.0-level
features, that's fine by me. That limits us to a maximum of six queries.
Just because we keep support for dead versions doesn't mean we have to
retroactively develop for them. Au contraire.

I'll make another patch version, with these few differences.

I managed to do testing against servers on all the versions >= 8.0 and
I discovered that prior to version 8.1 they don't support ALL/ANY on
oidvectors; so I added another query form.

But I don't like having so many different queries, so I am in favour
of trimming it down. Can I leave that up to the committer to remove
some cases or do we need another patch?

Great!

Here's another.

Attachments:

psql-select-tab-completion-v3.patchapplication/octet-stream; name=psql-select-tab-completion-v3.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8bc4a19..ae19c4f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1030,6 +1030,82 @@ static const SchemaQuery Query_for_list_of_statistics = {
 "       and pg_catalog.pg_table_is_visible(c2.oid)"\
 "       and c2.relispartition = 'true'"
 
+/* For server versions prior to 8.1, oidvector does not support op ALL/ANY.
+   This query should work for server versions 7.3 and later; prior to that,
+   even pg_function_is_visible doesn't exist. */
+#define Query_for_list_of_selectable_functions_or_attributes_PRE_81 \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
+/* For server versions prior to 9.0, do not check tables referencing pg_proc. */
+#define Query_for_list_of_selectable_functions_or_attributes_PRE_90 \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"      AND 'internal'::regtype != ALL (proargtypes) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
+/* For server versions prior to 9.6, pg_aggregate lacks several columns for support functions. */
+#define Query_for_list_of_selectable_functions_or_attributes_PRE_96 \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"      AND 'internal'::regtype != ALL (proargtypes) "\
+"      AND oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "\
+"      AND oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn]) FROM pg_aggregate) "\
+"      AND oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "\
+"      AND oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "\
+"      AND oid NOT IN (SELECT castfunc FROM pg_cast) "\
+"      AND oid NOT IN (SELECT amproc FROM pg_amproc) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
+#define Query_for_list_of_selectable_functions_or_attributes \
+" SELECT DISTINCT expansion FROM ( "\
+"   SELECT pg_catalog.quote_ident(proname)||'(' AS expansion, proname AS name "\
+"     FROM pg_catalog.pg_proc "\
+"    WHERE pg_catalog.pg_function_is_visible(oid) "\
+"      AND prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "\
+"      AND 'internal'::regtype <> ALL (proargtypes) "\
+"      AND oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "\
+"      AND oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn,aggcombinefn,aggserialfn,aggdeserialfn,aggmtransfn,aggminvtransfn,aggmfinalfn]) FROM pg_aggregate) "\
+"      AND oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "\
+"      AND oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "\
+"      AND oid NOT IN (SELECT castfunc FROM pg_cast) "\
+"      AND oid NOT IN (SELECT amproc FROM pg_amproc) "\
+"    UNION ALL "\
+"   SELECT pg_catalog.quote_ident(attname), attname "\
+"     FROM pg_catalog.pg_attribute "\
+"    WHERE attnum > 0 "\
+"      AND NOT attisdropped "\
+"   ) ss "\
+"  WHERE substring(pg_catalog.quote_ident(name),1,%d)='%s'"
+
 /*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
@@ -3247,7 +3323,18 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if ((TailMatches1("SELECT") || TailMatches2("SELECT", "ALL|DISTINCT")) &&
+			 pset.sversion >= 70300)
+	{
+		if (pset.sversion < 80100)
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes_PRE_81);
+		else if (pset.sversion < 90000)
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes_PRE_90);
+		else if (pset.sversion < 90600)
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes_PRE_96);
+		else
+			COMPLETE_WITH_QUERY(Query_for_list_of_selectable_functions_or_attributes);
+	}
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
#20Edmund Horner
ejrh00@gmail.com
In reply to: Vik Fearing (#15)
Re: PATCH: psql tab completion for SELECT

On 19 January 2018 at 05:37, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

On 01/18/2018 01:07 AM, Tom Lane wrote:

Edmund Horner <ejrh00@gmail.com> writes:

On 15 January 2018 at 15:45, Andres Freund <andres@anarazel.de> wrote:

All worries like this are supposed to check the server version.

In psql there are around 200 such tab completion queries, none of
which checks the server version. Many would cause the user's
transaction to abort if invoked on an older server. Identifying the
appropriate server versions for each one would be quite a bit of work.

Is there a better way to make this more robust?

Maybe it'd be worth the effort to wrap tab completion queries in
SAVEPOINT/RELEASE SAVEPOINT if we're inside a user transaction
(which we could detect from libpq's state, I believe).

That seems like an independent patch, but it'd be a prerequisite
if you want to issue tab completion queries with version dependencies.

A bigger point here is: do you really want SELECT tab completion
to work only against the latest and greatest server version?
That would become an argument against committing the feature at all;
maybe not enough to tip the scales against it, but still a demerit.

I don't really want such a patch. I use new psql on old servers all the
time.

I'm not sure where we got with this. I really should have put this
patch in a separate thread, since it's an independent feature.

The patch mentioned attempts to put savepoints around the tab
completion query where appropriate.

#21Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#20)
Re: PATCH: psql tab completion for SELECT

On 01/26/2018 01:28 AM, Edmund Horner wrote:

On 19 January 2018 at 05:37, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

On 01/18/2018 01:07 AM, Tom Lane wrote:

Edmund Horner <ejrh00@gmail.com> writes:

On 15 January 2018 at 15:45, Andres Freund <andres@anarazel.de> wrote:

All worries like this are supposed to check the server version.

In psql there are around 200 such tab completion queries, none of
which checks the server version. Many would cause the user's
transaction to abort if invoked on an older server. Identifying the
appropriate server versions for each one would be quite a bit of work.

Is there a better way to make this more robust?

Maybe it'd be worth the effort to wrap tab completion queries in
SAVEPOINT/RELEASE SAVEPOINT if we're inside a user transaction
(which we could detect from libpq's state, I believe).

That seems like an independent patch, but it'd be a prerequisite
if you want to issue tab completion queries with version dependencies.

A bigger point here is: do you really want SELECT tab completion
to work only against the latest and greatest server version?
That would become an argument against committing the feature at all;
maybe not enough to tip the scales against it, but still a demerit.

I don't really want such a patch. I use new psql on old servers all the
time.

I'm not sure where we got with this. I really should have put this
patch in a separate thread, since it's an independent feature.

Yes, it should have been a separate thread.

The patch mentioned attempts to put savepoints around the tab
completion query where appropriate.

I am -1 on this idea.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#22Edmund Horner
ejrh00@gmail.com
In reply to: Vik Fearing (#21)
Re: PATCH: psql tab completion for SELECT

On 26 January 2018 at 13:44, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

On 01/26/2018 01:28 AM, Edmund Horner wrote:

The patch mentioned attempts to put savepoints around the tab
completion query where appropriate.

I am -1 on this idea.

May I ask why? It doesn't stop psql working against older versions,
as it checks that the server supports savepoints.

#23Tom Lane
tgl@sss.pgh.pa.us
In reply to: Edmund Horner (#22)
1 attachment(s)
Re: PATCH: psql tab completion for SELECT

Edmund Horner <ejrh00@gmail.com> writes:

On 26 January 2018 at 13:44, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

On 01/26/2018 01:28 AM, Edmund Horner wrote:

The patch mentioned attempts to put savepoints around the tab
completion query where appropriate.

I am -1 on this idea.

May I ask why? It doesn't stop psql working against older versions,
as it checks that the server supports savepoints.

I looked into this patch and was disappointed to discover that it had
only a very ad-hoc solution to the problem of version-dependent tab
completion queries. We need something better --- in particular, the
recent prokind changes mean that there needs to be a way to make
SchemaQuery queries version-dependent.

So ... here is a modest proposal. It invents a VersionedQuery concept
and also extends the SchemaQuery infrastructure to allow those to be
versioned. I have not taken this nearly as far as it could be taken,
since it's mostly just proposing mechanism. To illustrate the
VersionedQuery infrastructure, I fixed it so it wouldn't send
publication/subscription queries to pre-v10 servers, and to illustrate
the versioned SchemaQuery infrastructure, I fixed the prokind problems.

If people like this approach, I propose to commit this more or less
as-is. The select-tab-completion patch would then need to be rewritten
to use this infrastructure, but I think that should be straightforward.
As a separate line of work, the infrastructure could be applied to fix
the pre-existing places where tab completion fails against old servers.
But that's probably work for v12 or beyond, unless somebody's really
motivated to do it right now.

regards, tom lane

Attachments:

version-dependent-tab-completion-1.patchtext/x-diff; charset=us-ascii; name=version-dependent-tab-completion-1.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 47909ed..9d0d45b 100644
*** a/src/bin/psql/tab-complete.c
--- b/src/bin/psql/tab-complete.c
*************** extern char *filename_completion_functio
*** 71,85 ****
--- 71,112 ----
  PQExpBuffer tab_completion_query_buf = NULL;
  
  /*
+  * In some situations, the query to find out what names are available to
+  * complete with must vary depending on server version.  We handle this by
+  * storing a list of queries, each tagged with the minimum server version
+  * it will work for.  Each list must be stored in descending server version
+  * order, so that the first satisfactory query is the one to use.
+  *
+  * When the query string is otherwise constant, an array of VersionedQuery
+  * suffices.  Terminate the array with 0/NULL.  (If the search reaches that
+  * entry, we give up and do no completion.)
+  */
+ typedef struct VersionedQuery
+ {
+ 	int			min_server_version;
+ 	const char *query;
+ } VersionedQuery;
+ 
+ /*
   * This struct is used to define "schema queries", which are custom-built
   * to obtain possibly-schema-qualified names of database objects.  There is
   * enough similarity in the structure that we don't want to repeat it each
   * time.  So we put the components of each query into this struct and
   * assemble them with the common boilerplate in _complete_from_query().
+  *
+  * As with VersionedQuery, we can use an array of these if the query details
+  * must vary across versions.
   */
  typedef struct SchemaQuery
  {
  	/*
+ 	 * If not zero, minimum server version this struct applies to.  If not
+ 	 * zero, there should be a following struct with a smaller minimum server
+ 	 * version; use catname == NULL in the last entry if it should do nothing.
+ 	 */
+ 	int			min_server_version;
+ 
+ 	/*
  	 * Name of catalog or catalogs to be queried, with alias, eg.
  	 * "pg_catalog.pg_class c".  Note that "pg_namespace n" will be added.
  	 */
*************** static const char *completion_charp;	/* 
*** 133,138 ****
--- 160,166 ----
  static const char *const *completion_charpp;	/* to pass a list of strings */
  static const char *completion_info_charp;	/* to pass a second string */
  static const char *completion_info_charp2;	/* to pass a third string */
+ static const VersionedQuery *completion_vquery; /* to pass a VersionedQuery */
  static const SchemaQuery *completion_squery;	/* to pass a SchemaQuery */
  static bool completion_case_sensitive;	/* completion is case sensitive */
  
*************** static bool completion_case_sensitive;	/
*** 140,146 ****
--- 168,176 ----
   * A few macros to ease typing. You can use these to complete the given
   * string with
   * 1) The results from a query you pass it. (Perhaps one of those below?)
+  *	  We support both simple and versioned queries.
   * 2) The results from a schema query you pass it.
+  *	  We support both simple and versioned schema queries.
   * 3) The items from a null-pointer-terminated list (with or without
   *	  case-sensitive comparison; see also COMPLETE_WITH_LISTn, below).
   * 4) A string constant.
*************** do { \
*** 153,158 ****
--- 183,194 ----
  	matches = completion_matches(text, complete_from_query); \
  } while (0)
  
+ #define COMPLETE_WITH_VERSIONED_QUERY(query) \
+ do { \
+ 	completion_vquery = query; \
+ 	matches = completion_matches(text, complete_from_versioned_query); \
+ } while (0)
+ 
  #define COMPLETE_WITH_SCHEMA_QUERY(query, addon) \
  do { \
  	completion_squery = &(query); \
*************** do { \
*** 160,165 ****
--- 196,208 ----
  	matches = completion_matches(text, complete_from_schema_query); \
  } while (0)
  
+ #define COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(query, addon) \
+ do { \
+ 	completion_squery = query; \
+ 	completion_vquery = addon; \
+ 	matches = completion_matches(text, complete_from_versioned_schema_query); \
+ } while (0)
+ 
  #define COMPLETE_WITH_LIST_CS(list) \
  do { \
  	completion_charpp = list; \
*************** do { \
*** 345,366 ****
   * Assembly instructions for schema queries
   */
  
! static const SchemaQuery Query_for_list_of_aggregates = {
! 	/* catname */
! 	"pg_catalog.pg_proc p",
! 	/* selcondition */
! 	"p.prokind = 'a'",
! 	/* viscondition */
! 	"pg_catalog.pg_function_is_visible(p.oid)",
! 	/* namespace */
! 	"p.pronamespace",
! 	/* result */
! 	"pg_catalog.quote_ident(p.proname)",
! 	/* qualresult */
! 	NULL
  };
  
  static const SchemaQuery Query_for_list_of_datatypes = {
  	/* catname */
  	"pg_catalog.pg_type t",
  	/* selcondition --- ignore table rowtypes and array types */
--- 388,431 ----
   * Assembly instructions for schema queries
   */
  
! static const SchemaQuery Query_for_list_of_aggregates[] = {
! 	{
! 		/* min_server_version */
! 		110000,
! 		/* catname */
! 		"pg_catalog.pg_proc p",
! 		/* selcondition */
! 		"p.prokind = 'a'",
! 		/* viscondition */
! 		"pg_catalog.pg_function_is_visible(p.oid)",
! 		/* namespace */
! 		"p.pronamespace",
! 		/* result */
! 		"pg_catalog.quote_ident(p.proname)",
! 		/* qualresult */
! 		NULL
! 	},
! 	{
! 		/* min_server_version */
! 		0,
! 		/* catname */
! 		"pg_catalog.pg_proc p",
! 		/* selcondition */
! 		"p.proisagg",
! 		/* viscondition */
! 		"pg_catalog.pg_function_is_visible(p.oid)",
! 		/* namespace */
! 		"p.pronamespace",
! 		/* result */
! 		"pg_catalog.quote_ident(p.proname)",
! 		/* qualresult */
! 		NULL
! 	}
  };
  
  static const SchemaQuery Query_for_list_of_datatypes = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_type t",
  	/* selcondition --- ignore table rowtypes and array types */
*************** static const SchemaQuery Query_for_list_
*** 379,384 ****
--- 444,451 ----
  };
  
  static const SchemaQuery Query_for_list_of_domains = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_type t",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 393,414 ****
  	NULL
  };
  
! static const SchemaQuery Query_for_list_of_functions = {
! 	/* catname */
! 	"pg_catalog.pg_proc p",
! 	/* selcondition */
! 	"p.prokind IN ('f', 'w')",
! 	/* viscondition */
! 	"pg_catalog.pg_function_is_visible(p.oid)",
! 	/* namespace */
! 	"p.pronamespace",
! 	/* result */
! 	"pg_catalog.quote_ident(p.proname)",
! 	/* qualresult */
! 	NULL
  };
  
  static const SchemaQuery Query_for_list_of_indexes = {
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
--- 460,503 ----
  	NULL
  };
  
! static const SchemaQuery Query_for_list_of_functions[] = {
! 	{
! 		/* min_server_version */
! 		110000,
! 		/* catname */
! 		"pg_catalog.pg_proc p",
! 		/* selcondition */
! 		"p.prokind IN ('f', 'w')",
! 		/* viscondition */
! 		"pg_catalog.pg_function_is_visible(p.oid)",
! 		/* namespace */
! 		"p.pronamespace",
! 		/* result */
! 		"pg_catalog.quote_ident(p.proname)",
! 		/* qualresult */
! 		NULL
! 	},
! 	{
! 		/* min_server_version */
! 		0,
! 		/* catname */
! 		"pg_catalog.pg_proc p",
! 		/* selcondition */
! 		NULL,
! 		/* viscondition */
! 		"pg_catalog.pg_function_is_visible(p.oid)",
! 		/* namespace */
! 		"p.pronamespace",
! 		/* result */
! 		"pg_catalog.quote_ident(p.proname)",
! 		/* qualresult */
! 		NULL
! 	}
  };
  
  static const SchemaQuery Query_for_list_of_indexes = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 424,445 ****
  	NULL
  };
  
! static const SchemaQuery Query_for_list_of_procedures = {
! 	/* catname */
! 	"pg_catalog.pg_proc p",
! 	/* selcondition */
! 	"p.prokind = 'p'",
! 	/* viscondition */
! 	"pg_catalog.pg_function_is_visible(p.oid)",
! 	/* namespace */
! 	"p.pronamespace",
! 	/* result */
! 	"pg_catalog.quote_ident(p.proname)",
! 	/* qualresult */
! 	NULL
  };
  
  static const SchemaQuery Query_for_list_of_routines = {
  	/* catname */
  	"pg_catalog.pg_proc p",
  	/* selcondition */
--- 513,541 ----
  	NULL
  };
  
! static const SchemaQuery Query_for_list_of_procedures[] = {
! 	{
! 		/* min_server_version */
! 		110000,
! 		/* catname */
! 		"pg_catalog.pg_proc p",
! 		/* selcondition */
! 		"p.prokind = 'p'",
! 		/* viscondition */
! 		"pg_catalog.pg_function_is_visible(p.oid)",
! 		/* namespace */
! 		"p.pronamespace",
! 		/* result */
! 		"pg_catalog.quote_ident(p.proname)",
! 		/* qualresult */
! 		NULL
! 	},
! 	{0, NULL}
  };
  
  static const SchemaQuery Query_for_list_of_routines = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_proc p",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 455,460 ****
--- 551,558 ----
  };
  
  static const SchemaQuery Query_for_list_of_sequences = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 470,475 ****
--- 568,575 ----
  };
  
  static const SchemaQuery Query_for_list_of_foreign_tables = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 485,490 ****
--- 585,592 ----
  };
  
  static const SchemaQuery Query_for_list_of_tables = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 501,506 ****
--- 603,610 ----
  };
  
  static const SchemaQuery Query_for_list_of_partitioned_tables = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 516,521 ****
--- 620,627 ----
  };
  
  static const SchemaQuery Query_for_list_of_constraints_with_schema = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_constraint c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 532,537 ****
--- 638,645 ----
  
  /* Relations supporting INSERT, UPDATE or DELETE */
  static const SchemaQuery Query_for_list_of_updatables = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 550,555 ****
--- 658,665 ----
  };
  
  static const SchemaQuery Query_for_list_of_relations = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 565,570 ****
--- 675,682 ----
  };
  
  static const SchemaQuery Query_for_list_of_tsvmf = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 585,590 ****
--- 697,704 ----
  };
  
  static const SchemaQuery Query_for_list_of_tmf = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 602,607 ****
--- 716,723 ----
  };
  
  static const SchemaQuery Query_for_list_of_tpm = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 619,624 ****
--- 735,742 ----
  };
  
  static const SchemaQuery Query_for_list_of_tm = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 635,640 ****
--- 753,760 ----
  };
  
  static const SchemaQuery Query_for_list_of_views = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 650,655 ****
--- 770,777 ----
  };
  
  static const SchemaQuery Query_for_list_of_matviews = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_class c",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 665,670 ****
--- 787,794 ----
  };
  
  static const SchemaQuery Query_for_list_of_statistics = {
+ 	/* min_server_version */
+ 	0,
  	/* catname */
  	"pg_catalog.pg_statistic_ext s",
  	/* selcondition */
*************** static const SchemaQuery Query_for_list_
*** 925,942 ****
  "   FROM pg_catalog.pg_am "\
  "  WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s'"
  
- #define Query_for_list_of_publications \
- " SELECT pg_catalog.quote_ident(pubname) "\
- "   FROM pg_catalog.pg_publication "\
- "  WHERE substring(pg_catalog.quote_ident(pubname),1,%d)='%s'"
- 
- #define Query_for_list_of_subscriptions \
- " SELECT pg_catalog.quote_ident(s.subname) "\
- "   FROM pg_catalog.pg_subscription s, pg_catalog.pg_database d "\
- "  WHERE substring(pg_catalog.quote_ident(s.subname),1,%d)='%s' "\
- "    AND d.datname = pg_catalog.current_database() "\
- "    AND s.subdbid = d.oid"
- 
  /* the silly-looking length condition is just to eat up the current word */
  #define Query_for_list_of_arguments \
  "SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
--- 1049,1054 ----
*************** static const SchemaQuery Query_for_list_
*** 1031,1036 ****
--- 1143,1174 ----
  "       and c2.relispartition = 'true'"
  
  /*
+  * These object types were introduced later than our support cutoff of
+  * server version 7.4.  We use the VersionedQuery infrastructure so that
+  * we don't send certain-to-fail queries to older servers.
+  */
+ 
+ static const VersionedQuery Query_for_list_of_publications[] = {
+ 	{100000,
+ 		" SELECT pg_catalog.quote_ident(pubname) "
+ 		"   FROM pg_catalog.pg_publication "
+ 		"  WHERE substring(pg_catalog.quote_ident(pubname),1,%d)='%s'"
+ 	},
+ 	{0, NULL}
+ };
+ 
+ static const VersionedQuery Query_for_list_of_subscriptions[] = {
+ 	{100000,
+ 		" SELECT pg_catalog.quote_ident(s.subname) "
+ 		"   FROM pg_catalog.pg_subscription s, pg_catalog.pg_database d "
+ 		"  WHERE substring(pg_catalog.quote_ident(s.subname),1,%d)='%s' "
+ 		"    AND d.datname = pg_catalog.current_database() "
+ 		"    AND s.subdbid = d.oid"
+ 	},
+ 	{0, NULL}
+ };
+ 
+ /*
   * This is a list of all "things" in Pgsql, which can show up after CREATE or
   * DROP; and there is also a query to get a list of them.
   */
*************** typedef struct
*** 1039,1044 ****
--- 1177,1183 ----
  {
  	const char *name;
  	const char *query;			/* simple query, or NULL */
+ 	const VersionedQuery *vquery;	/* versioned query, or NULL */
  	const SchemaQuery *squery;	/* schema query, or NULL */
  	const bits32 flags;			/* visibility flags, see below */
  } pgsql_thing_t;
*************** typedef struct
*** 1049,1057 ****
  #define THING_NO_SHOW		(THING_NO_CREATE | THING_NO_DROP | THING_NO_ALTER)
  
  static const pgsql_thing_t words_after_create[] = {
! 	{"ACCESS METHOD", NULL, NULL, THING_NO_ALTER},
! 	{"AGGREGATE", NULL, &Query_for_list_of_aggregates},
! 	{"CAST", NULL, NULL},		/* Casts have complex structures for names, so
  								 * skip it */
  	{"COLLATION", "SELECT pg_catalog.quote_ident(collname) FROM pg_catalog.pg_collation WHERE collencoding IN (-1, pg_catalog.pg_char_to_encoding(pg_catalog.getdatabaseencoding())) AND substring(pg_catalog.quote_ident(collname),1,%d)='%s'"},
  
--- 1188,1196 ----
  #define THING_NO_SHOW		(THING_NO_CREATE | THING_NO_DROP | THING_NO_ALTER)
  
  static const pgsql_thing_t words_after_create[] = {
! 	{"ACCESS METHOD", NULL, NULL, NULL, THING_NO_ALTER},
! 	{"AGGREGATE", NULL, NULL, Query_for_list_of_aggregates},
! 	{"CAST", NULL, NULL, NULL}, /* Casts have complex structures for names, so
  								 * skip it */
  	{"COLLATION", "SELECT pg_catalog.quote_ident(collname) FROM pg_catalog.pg_collation WHERE collencoding IN (-1, pg_catalog.pg_char_to_encoding(pg_catalog.getdatabaseencoding())) AND substring(pg_catalog.quote_ident(collname),1,%d)='%s'"},
  
*************** static const pgsql_thing_t words_after_c
*** 1059,1114 ****
  	 * CREATE CONSTRAINT TRIGGER is not supported here because it is designed
  	 * to be used only by pg_dump.
  	 */
! 	{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
  	{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
  	{"DATABASE", Query_for_list_of_databases},
! 	{"DEFAULT PRIVILEGES", NULL, NULL, THING_NO_CREATE | THING_NO_DROP},
! 	{"DICTIONARY", Query_for_list_of_ts_dictionaries, NULL, THING_NO_SHOW},
! 	{"DOMAIN", NULL, &Query_for_list_of_domains},
! 	{"EVENT TRIGGER", NULL, NULL},
  	{"EXTENSION", Query_for_list_of_extensions},
! 	{"FOREIGN DATA WRAPPER", NULL, NULL},
! 	{"FOREIGN TABLE", NULL, NULL},
! 	{"FUNCTION", NULL, &Query_for_list_of_functions},
  	{"GROUP", Query_for_list_of_roles},
! 	{"INDEX", NULL, &Query_for_list_of_indexes},
  	{"LANGUAGE", Query_for_list_of_languages},
! 	{"LARGE OBJECT", NULL, NULL, THING_NO_CREATE | THING_NO_DROP},
! 	{"MATERIALIZED VIEW", NULL, &Query_for_list_of_matviews},
! 	{"OPERATOR", NULL, NULL},	/* Querying for this is probably not such a
! 								 * good idea. */
! 	{"OWNED", NULL, NULL, THING_NO_CREATE | THING_NO_ALTER},	/* for DROP OWNED BY ... */
! 	{"PARSER", Query_for_list_of_ts_parsers, NULL, THING_NO_SHOW},
! 	{"POLICY", NULL, NULL},
! 	{"PROCEDURE", NULL, &Query_for_list_of_procedures},
! 	{"PUBLICATION", Query_for_list_of_publications},
  	{"ROLE", Query_for_list_of_roles},
! 	{"ROUTINE", NULL, &Query_for_list_of_routines, THING_NO_CREATE},
  	{"RULE", "SELECT pg_catalog.quote_ident(rulename) FROM pg_catalog.pg_rules WHERE substring(pg_catalog.quote_ident(rulename),1,%d)='%s'"},
  	{"SCHEMA", Query_for_list_of_schemas},
! 	{"SEQUENCE", NULL, &Query_for_list_of_sequences},
  	{"SERVER", Query_for_list_of_servers},
! 	{"STATISTICS", NULL, &Query_for_list_of_statistics},
! 	{"SUBSCRIPTION", Query_for_list_of_subscriptions},
! 	{"SYSTEM", NULL, NULL, THING_NO_CREATE | THING_NO_DROP},
! 	{"TABLE", NULL, &Query_for_list_of_tables},
  	{"TABLESPACE", Query_for_list_of_tablespaces},
! 	{"TEMP", NULL, NULL, THING_NO_DROP | THING_NO_ALTER},	/* for CREATE TEMP TABLE
! 															 * ... */
! 	{"TEMPLATE", Query_for_list_of_ts_templates, NULL, THING_NO_SHOW},
! 	{"TEMPORARY", NULL, NULL, THING_NO_DROP | THING_NO_ALTER},	/* for CREATE TEMPORARY
! 																 * TABLE ... */
! 	{"TEXT SEARCH", NULL, NULL},
! 	{"TRANSFORM", NULL, NULL},
  	{"TRIGGER", "SELECT pg_catalog.quote_ident(tgname) FROM pg_catalog.pg_trigger WHERE substring(pg_catalog.quote_ident(tgname),1,%d)='%s' AND NOT tgisinternal"},
! 	{"TYPE", NULL, &Query_for_list_of_datatypes},
! 	{"UNIQUE", NULL, NULL, THING_NO_DROP | THING_NO_ALTER}, /* for CREATE UNIQUE
! 															 * INDEX ... */
! 	{"UNLOGGED", NULL, NULL, THING_NO_DROP | THING_NO_ALTER},	/* for CREATE UNLOGGED
! 																 * TABLE ... */
  	{"USER", Query_for_list_of_roles " UNION SELECT 'MAPPING FOR'"},
! 	{"USER MAPPING FOR", NULL, NULL},
! 	{"VIEW", NULL, &Query_for_list_of_views},
  	{NULL}						/* end of list */
  };
  
--- 1198,1253 ----
  	 * CREATE CONSTRAINT TRIGGER is not supported here because it is designed
  	 * to be used only by pg_dump.
  	 */
! 	{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, NULL, THING_NO_SHOW},
  	{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
  	{"DATABASE", Query_for_list_of_databases},
! 	{"DEFAULT PRIVILEGES", NULL, NULL, NULL, THING_NO_CREATE | THING_NO_DROP},
! 	{"DICTIONARY", Query_for_list_of_ts_dictionaries, NULL, NULL, THING_NO_SHOW},
! 	{"DOMAIN", NULL, NULL, &Query_for_list_of_domains},
! 	{"EVENT TRIGGER", NULL, NULL, NULL},
  	{"EXTENSION", Query_for_list_of_extensions},
! 	{"FOREIGN DATA WRAPPER", NULL, NULL, NULL},
! 	{"FOREIGN TABLE", NULL, NULL, NULL},
! 	{"FUNCTION", NULL, NULL, Query_for_list_of_functions},
  	{"GROUP", Query_for_list_of_roles},
! 	{"INDEX", NULL, NULL, &Query_for_list_of_indexes},
  	{"LANGUAGE", Query_for_list_of_languages},
! 	{"LARGE OBJECT", NULL, NULL, NULL, THING_NO_CREATE | THING_NO_DROP},
! 	{"MATERIALIZED VIEW", NULL, NULL, &Query_for_list_of_matviews},
! 	{"OPERATOR", NULL, NULL, NULL}, /* Querying for this is probably not such
! 									 * a good idea. */
! 	{"OWNED", NULL, NULL, NULL, THING_NO_CREATE | THING_NO_ALTER},	/* for DROP OWNED BY ... */
! 	{"PARSER", Query_for_list_of_ts_parsers, NULL, NULL, THING_NO_SHOW},
! 	{"POLICY", NULL, NULL, NULL},
! 	{"PROCEDURE", NULL, NULL, Query_for_list_of_procedures},
! 	{"PUBLICATION", NULL, Query_for_list_of_publications},
  	{"ROLE", Query_for_list_of_roles},
! 	{"ROUTINE", NULL, NULL, &Query_for_list_of_routines, THING_NO_CREATE},
  	{"RULE", "SELECT pg_catalog.quote_ident(rulename) FROM pg_catalog.pg_rules WHERE substring(pg_catalog.quote_ident(rulename),1,%d)='%s'"},
  	{"SCHEMA", Query_for_list_of_schemas},
! 	{"SEQUENCE", NULL, NULL, &Query_for_list_of_sequences},
  	{"SERVER", Query_for_list_of_servers},
! 	{"STATISTICS", NULL, NULL, &Query_for_list_of_statistics},
! 	{"SUBSCRIPTION", NULL, Query_for_list_of_subscriptions},
! 	{"SYSTEM", NULL, NULL, NULL, THING_NO_CREATE | THING_NO_DROP},
! 	{"TABLE", NULL, NULL, &Query_for_list_of_tables},
  	{"TABLESPACE", Query_for_list_of_tablespaces},
! 	{"TEMP", NULL, NULL, NULL, THING_NO_DROP | THING_NO_ALTER}, /* for CREATE TEMP TABLE
! 																 * ... */
! 	{"TEMPLATE", Query_for_list_of_ts_templates, NULL, NULL, THING_NO_SHOW},
! 	{"TEMPORARY", NULL, NULL, NULL, THING_NO_DROP | THING_NO_ALTER},	/* for CREATE TEMPORARY
! 																		 * TABLE ... */
! 	{"TEXT SEARCH", NULL, NULL, NULL},
! 	{"TRANSFORM", NULL, NULL, NULL},
  	{"TRIGGER", "SELECT pg_catalog.quote_ident(tgname) FROM pg_catalog.pg_trigger WHERE substring(pg_catalog.quote_ident(tgname),1,%d)='%s' AND NOT tgisinternal"},
! 	{"TYPE", NULL, NULL, &Query_for_list_of_datatypes},
! 	{"UNIQUE", NULL, NULL, NULL, THING_NO_DROP | THING_NO_ALTER},	/* for CREATE UNIQUE
! 																	 * INDEX ... */
! 	{"UNLOGGED", NULL, NULL, NULL, THING_NO_DROP | THING_NO_ALTER}, /* for CREATE UNLOGGED
! 																	 * TABLE ... */
  	{"USER", Query_for_list_of_roles " UNION SELECT 'MAPPING FOR'"},
! 	{"USER MAPPING FOR", NULL, NULL, NULL},
! 	{"VIEW", NULL, NULL, &Query_for_list_of_views},
  	{NULL}						/* end of list */
  };
  
*************** static char *create_command_generator(co
*** 1119,1126 ****
  static char *drop_command_generator(const char *text, int state);
  static char *alter_command_generator(const char *text, int state);
  static char *complete_from_query(const char *text, int state);
  static char *complete_from_schema_query(const char *text, int state);
! static char *_complete_from_query(int is_schema_query,
  					 const char *text, int state);
  static char *complete_from_list(const char *text, int state);
  static char *complete_from_const(const char *text, int state);
--- 1258,1268 ----
  static char *drop_command_generator(const char *text, int state);
  static char *alter_command_generator(const char *text, int state);
  static char *complete_from_query(const char *text, int state);
+ static char *complete_from_versioned_query(const char *text, int state);
  static char *complete_from_schema_query(const char *text, int state);
! static char *complete_from_versioned_schema_query(const char *text, int state);
! static char *_complete_from_query(const char *simple_query,
! 					 const SchemaQuery *schema_query,
  					 const char *text, int state);
  static char *complete_from_list(const char *text, int state);
  static char *complete_from_const(const char *text, int state);
*************** psql_completion(const char *text, int st
*** 2208,2214 ****
  		COMPLETE_WITH_LIST4("WORK", "TRANSACTION", "TO SAVEPOINT", "PREPARED");
  /* CALL */
  	else if (Matches1("CALL"))
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_procedures, NULL);
  	else if (Matches2("CALL", MatchAny))
  		COMPLETE_WITH_CONST("(");
  /* CLUSTER */
--- 2350,2356 ----
  		COMPLETE_WITH_LIST4("WORK", "TRANSACTION", "TO SAVEPOINT", "PREPARED");
  /* CALL */
  	else if (Matches1("CALL"))
! 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_procedures, NULL);
  	else if (Matches2("CALL", MatchAny))
  		COMPLETE_WITH_CONST("(");
  /* CLUSTER */
*************** psql_completion(const char *text, int st
*** 2654,2660 ****
  	else if (HeadMatches2("CREATE", "TRIGGER") && TailMatches1("EXECUTE"))
  		COMPLETE_WITH_CONST("PROCEDURE");
  	else if (HeadMatches2("CREATE", "TRIGGER") && TailMatches2("EXECUTE", "PROCEDURE"))
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  
  /* CREATE ROLE,USER,GROUP <name> */
  	else if (Matches3("CREATE", "ROLE|GROUP|USER", MatchAny) &&
--- 2796,2802 ----
  	else if (HeadMatches2("CREATE", "TRIGGER") && TailMatches1("EXECUTE"))
  		COMPLETE_WITH_CONST("PROCEDURE");
  	else if (HeadMatches2("CREATE", "TRIGGER") && TailMatches2("EXECUTE", "PROCEDURE"))
! 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  
  /* CREATE ROLE,USER,GROUP <name> */
  	else if (Matches3("CREATE", "ROLE|GROUP|USER", MatchAny) &&
*************** psql_completion(const char *text, int st
*** 3008,3018 ****
  		else if (TailMatches1("DOMAIN"))
  			COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
  		else if (TailMatches1("FUNCTION"))
! 			COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  		else if (TailMatches1("LANGUAGE"))
  			COMPLETE_WITH_QUERY(Query_for_list_of_languages);
  		else if (TailMatches1("PROCEDURE"))
! 			COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_procedures, NULL);
  		else if (TailMatches1("ROUTINE"))
  			COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_routines, NULL);
  		else if (TailMatches1("SCHEMA"))
--- 3150,3160 ----
  		else if (TailMatches1("DOMAIN"))
  			COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
  		else if (TailMatches1("FUNCTION"))
! 			COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  		else if (TailMatches1("LANGUAGE"))
  			COMPLETE_WITH_QUERY(Query_for_list_of_languages);
  		else if (TailMatches1("PROCEDURE"))
! 			COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_procedures, NULL);
  		else if (TailMatches1("ROUTINE"))
  			COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_routines, NULL);
  		else if (TailMatches1("SCHEMA"))
*************** psql_completion(const char *text, int st
*** 3483,3489 ****
  			COMPLETE_WITH_QUERY(Query_for_list_of_roles);
  	}
  	else if (TailMatchesCS1("\\da*"))
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_aggregates, NULL);
  	else if (TailMatchesCS1("\\dA*"))
  		COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
  	else if (TailMatchesCS1("\\db*"))
--- 3625,3631 ----
  			COMPLETE_WITH_QUERY(Query_for_list_of_roles);
  	}
  	else if (TailMatchesCS1("\\da*"))
! 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_aggregates, NULL);
  	else if (TailMatchesCS1("\\dA*"))
  		COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
  	else if (TailMatchesCS1("\\db*"))
*************** psql_completion(const char *text, int st
*** 3497,3503 ****
  	else if (TailMatchesCS1("\\dew*"))
  		COMPLETE_WITH_QUERY(Query_for_list_of_fdws);
  	else if (TailMatchesCS1("\\df*"))
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  
  	else if (TailMatchesCS1("\\dFd*"))
  		COMPLETE_WITH_QUERY(Query_for_list_of_ts_dictionaries);
--- 3639,3645 ----
  	else if (TailMatchesCS1("\\dew*"))
  		COMPLETE_WITH_QUERY(Query_for_list_of_fdws);
  	else if (TailMatchesCS1("\\df*"))
! 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  
  	else if (TailMatchesCS1("\\dFd*"))
  		COMPLETE_WITH_QUERY(Query_for_list_of_ts_dictionaries);
*************** psql_completion(const char *text, int st
*** 3541,3547 ****
  		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_relations, NULL);
  
  	else if (TailMatchesCS1("\\ef"))
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  	else if (TailMatchesCS1("\\ev"))
  		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
  
--- 3683,3689 ----
  		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_relations, NULL);
  
  	else if (TailMatchesCS1("\\ef"))
! 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  	else if (TailMatchesCS1("\\ev"))
  		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
  
*************** psql_completion(const char *text, int st
*** 3650,3656 ****
  			COMPLETE_WITH_LIST_CS3("default", "verbose", "terse");
  	}
  	else if (TailMatchesCS1("\\sf*"))
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  	else if (TailMatchesCS1("\\sv*"))
  		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
  	else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\i|\\include|"
--- 3792,3798 ----
  			COMPLETE_WITH_LIST_CS3("default", "verbose", "terse");
  	}
  	else if (TailMatchesCS1("\\sf*"))
! 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
  	else if (TailMatchesCS1("\\sv*"))
  		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
  	else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\i|\\include|"
*************** psql_completion(const char *text, int st
*** 3676,3684 ****
  			{
  				if (words_after_create[i].query)
  					COMPLETE_WITH_QUERY(words_after_create[i].query);
  				else if (words_after_create[i].squery)
! 					COMPLETE_WITH_SCHEMA_QUERY(*words_after_create[i].squery,
! 											   NULL);
  				break;
  			}
  		}
--- 3818,3828 ----
  			{
  				if (words_after_create[i].query)
  					COMPLETE_WITH_QUERY(words_after_create[i].query);
+ 				else if (words_after_create[i].vquery)
+ 					COMPLETE_WITH_VERSIONED_QUERY(words_after_create[i].vquery);
  				else if (words_after_create[i].squery)
! 					COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(words_after_create[i].squery,
! 														 NULL);
  				break;
  			}
  		}
*************** alter_command_generator(const char *text
*** 3777,3800 ****
  	return create_or_drop_command_generator(text, state, THING_NO_ALTER);
  }
  
! /* The following two functions are wrappers for _complete_from_query */
  
  static char *
  complete_from_query(const char *text, int state)
  {
! 	return _complete_from_query(0, text, state);
  }
  
  static char *
  complete_from_schema_query(const char *text, int state)
  {
! 	return _complete_from_query(1, text, state);
  }
  
  
  /*
!  * This creates a list of matching things, according to a query pointed to
!  * by completion_charp.
   * The query can be one of two kinds:
   *
   * 1. A simple query which must contain a %d and a %s, which will be replaced
--- 3921,3993 ----
  	return create_or_drop_command_generator(text, state, THING_NO_ALTER);
  }
  
! /*
!  * These functions generate lists using server queries.
!  * They are all wrappers for _complete_from_query.
!  */
  
  static char *
  complete_from_query(const char *text, int state)
  {
! 	/* query is assumed to work for any server version */
! 	return _complete_from_query(completion_charp, NULL, text, state);
! }
! 
! static char *
! complete_from_versioned_query(const char *text, int state)
! {
! 	const VersionedQuery *vquery = completion_vquery;
! 
! 	/* Find appropriate array element */
! 	while (pset.sversion < vquery->min_server_version)
! 		vquery++;
! 	/* Fail completion if server is too old */
! 	if (vquery->query == NULL)
! 		return NULL;
! 
! 	return _complete_from_query(vquery->query, NULL, text, state);
  }
  
  static char *
  complete_from_schema_query(const char *text, int state)
  {
! 	/* query is assumed to work for any server version */
! 	return _complete_from_query(completion_charp, completion_squery,
! 								text, state);
! }
! 
! static char *
! complete_from_versioned_schema_query(const char *text, int state)
! {
! 	const SchemaQuery *squery = completion_squery;
! 	const VersionedQuery *vquery = completion_vquery;
! 
! 	/* Find appropriate array element */
! 	while (pset.sversion < squery->min_server_version)
! 		squery++;
! 	/* Fail completion if server is too old */
! 	if (squery->catname == NULL)
! 		return NULL;
! 
! 	/* Likewise for the add-on text, if any */
! 	if (vquery)
! 	{
! 		while (pset.sversion < vquery->min_server_version)
! 			vquery++;
! 		if (vquery->query == NULL)
! 			return NULL;
! 	}
! 
! 	return _complete_from_query(vquery ? vquery->query : NULL,
! 								squery, text, state);
  }
  
  
  /*
!  * This creates a list of matching things, according to a query described by
!  * the initial arguments.  The caller has already done any work needed to
!  * select the appropriate query for the server's version.
!  *
   * The query can be one of two kinds:
   *
   * 1. A simple query which must contain a %d and a %s, which will be replaced
*************** complete_from_schema_query(const char *t
*** 3808,3820 ****
   * %d %s %d %s %d %s %s %d %s
   * where %d is the string length of the text and %s the text itself.
   *
   * It is assumed that strings should be escaped to become SQL literals
   * (that is, what is in the query is actually ... '%s' ...)
   *
   * See top of file for examples of both kinds of query.
   */
  static char *
! _complete_from_query(int is_schema_query, const char *text, int state)
  {
  	static int	list_index,
  				byte_length;
--- 4001,4020 ----
   * %d %s %d %s %d %s %s %d %s
   * where %d is the string length of the text and %s the text itself.
   *
+  * If both simple_query and schema_query are non-NULL, then we construct
+  * a schema query and append the (uninterpreted) string simple_query to it.
+  *
   * It is assumed that strings should be escaped to become SQL literals
   * (that is, what is in the query is actually ... '%s' ...)
   *
   * See top of file for examples of both kinds of query.
+  *
+  * "text" and "state" are supplied by readline.
   */
  static char *
! _complete_from_query(const char *simple_query,
! 					 const SchemaQuery *schema_query,
! 					 const char *text, int state)
  {
  	static int	list_index,
  				byte_length;
*************** _complete_from_query(int is_schema_query
*** 3865,3890 ****
  
  		initPQExpBuffer(&query_buffer);
  
! 		if (is_schema_query)
  		{
! 			/* completion_squery gives us the pieces to assemble */
! 			const char *qualresult = completion_squery->qualresult;
  
  			if (qualresult == NULL)
! 				qualresult = completion_squery->result;
  
  			/* Get unqualified names matching the input-so-far */
  			appendPQExpBuffer(&query_buffer, "SELECT %s FROM %s WHERE ",
! 							  completion_squery->result,
! 							  completion_squery->catname);
! 			if (completion_squery->selcondition)
  				appendPQExpBuffer(&query_buffer, "%s AND ",
! 								  completion_squery->selcondition);
  			appendPQExpBuffer(&query_buffer, "substring(%s,1,%d)='%s'",
! 							  completion_squery->result,
  							  char_length, e_text);
  			appendPQExpBuffer(&query_buffer, " AND %s",
! 							  completion_squery->viscondition);
  
  			/*
  			 * When fetching relation names, suppress system catalogs unless
--- 4065,4090 ----
  
  		initPQExpBuffer(&query_buffer);
  
! 		if (schema_query)
  		{
! 			/* schema_query gives us the pieces to assemble */
! 			const char *qualresult = schema_query->qualresult;
  
  			if (qualresult == NULL)
! 				qualresult = schema_query->result;
  
  			/* Get unqualified names matching the input-so-far */
  			appendPQExpBuffer(&query_buffer, "SELECT %s FROM %s WHERE ",
! 							  schema_query->result,
! 							  schema_query->catname);
! 			if (schema_query->selcondition)
  				appendPQExpBuffer(&query_buffer, "%s AND ",
! 								  schema_query->selcondition);
  			appendPQExpBuffer(&query_buffer, "substring(%s,1,%d)='%s'",
! 							  schema_query->result,
  							  char_length, e_text);
  			appendPQExpBuffer(&query_buffer, " AND %s",
! 							  schema_query->viscondition);
  
  			/*
  			 * When fetching relation names, suppress system catalogs unless
*************** _complete_from_query(int is_schema_query
*** 3892,3898 ****
  			 * between not offering system catalogs for completion at all, and
  			 * having them swamp the result when the input is just "p".
  			 */
! 			if (strcmp(completion_squery->catname,
  					   "pg_catalog.pg_class c") == 0 &&
  				strncmp(text, "pg_", 3) !=0)
  			{
--- 4092,4098 ----
  			 * between not offering system catalogs for completion at all, and
  			 * having them swamp the result when the input is just "p".
  			 */
! 			if (strcmp(schema_query->catname,
  					   "pg_catalog.pg_class c") == 0 &&
  				strncmp(text, "pg_", 3) !=0)
  			{
*************** _complete_from_query(int is_schema_query
*** 3926,3936 ****
  							  "FROM %s, pg_catalog.pg_namespace n "
  							  "WHERE %s = n.oid AND ",
  							  qualresult,
! 							  completion_squery->catname,
! 							  completion_squery->namespace);
! 			if (completion_squery->selcondition)
  				appendPQExpBuffer(&query_buffer, "%s AND ",
! 								  completion_squery->selcondition);
  			appendPQExpBuffer(&query_buffer, "substring(pg_catalog.quote_ident(n.nspname) || '.' || %s,1,%d)='%s'",
  							  qualresult,
  							  char_length, e_text);
--- 4126,4136 ----
  							  "FROM %s, pg_catalog.pg_namespace n "
  							  "WHERE %s = n.oid AND ",
  							  qualresult,
! 							  schema_query->catname,
! 							  schema_query->namespace);
! 			if (schema_query->selcondition)
  				appendPQExpBuffer(&query_buffer, "%s AND ",
! 								  schema_query->selcondition);
  			appendPQExpBuffer(&query_buffer, "substring(pg_catalog.quote_ident(n.nspname) || '.' || %s,1,%d)='%s'",
  							  qualresult,
  							  char_length, e_text);
*************** _complete_from_query(int is_schema_query
*** 3951,3963 ****
  							  char_length, e_text);
  
  			/* If an addon query was provided, use it */
! 			if (completion_charp)
! 				appendPQExpBuffer(&query_buffer, "\n%s", completion_charp);
  		}
  		else
  		{
! 			/* completion_charp is an sprintf-style format string */
! 			appendPQExpBuffer(&query_buffer, completion_charp,
  							  char_length, e_text,
  							  e_info_charp, e_info_charp,
  							  e_info_charp2, e_info_charp2);
--- 4151,4164 ----
  							  char_length, e_text);
  
  			/* If an addon query was provided, use it */
! 			if (simple_query)
! 				appendPQExpBuffer(&query_buffer, "\n%s", simple_query);
  		}
  		else
  		{
! 			Assert(simple_query);
! 			/* simple_query is an sprintf-style format string */
! 			appendPQExpBuffer(&query_buffer, simple_query,
  							  char_length, e_text,
  							  e_info_charp, e_info_charp,
  							  e_info_charp2, e_info_charp2);
#24Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Tom Lane (#23)
Re: PATCH: psql tab completion for SELECT

On 03/04/2018 08:06 PM, Tom Lane wrote:

Edmund Horner <ejrh00@gmail.com> writes:

On 26 January 2018 at 13:44, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

On 01/26/2018 01:28 AM, Edmund Horner wrote:

The patch mentioned attempts to put savepoints around the tab
completion query where appropriate.

I am -1 on this idea.

May I ask why? It doesn't stop psql working against older versions,
as it checks that the server supports savepoints.

I looked into this patch and was disappointed to discover that it had
only a very ad-hoc solution to the problem of version-dependent tab
completion queries. We need something better --- in particular, the
recent prokind changes mean that there needs to be a way to make
SchemaQuery queries version-dependent.

So ... here is a modest proposal. It invents a VersionedQuery concept
and also extends the SchemaQuery infrastructure to allow those to be
versioned. I have not taken this nearly as far as it could be taken,
since it's mostly just proposing mechanism. To illustrate the
VersionedQuery infrastructure, I fixed it so it wouldn't send
publication/subscription queries to pre-v10 servers, and to illustrate
the versioned SchemaQuery infrastructure, I fixed the prokind problems.

If people like this approach, I propose to commit this more or less
as-is. The select-tab-completion patch would then need to be rewritten
to use this infrastructure, but I think that should be straightforward.
As a separate line of work, the infrastructure could be applied to fix
the pre-existing places where tab completion fails against old servers.
But that's probably work for v12 or beyond, unless somebody's really
motivated to do it right now.

Tab completion on functions in SELECT (a subset of this thread's patch)
is quite important to me personally. I will work on this in the coming
days.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#25Edmund Horner
ejrh00@gmail.com
In reply to: Tom Lane (#23)
Re: PATCH: psql tab completion for SELECT

On 5 March 2018 at 08:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I looked into this patch and was disappointed to discover that it had
only a very ad-hoc solution to the problem of version-dependent tab
completion queries. We need something better --- in particular, the
recent prokind changes mean that there needs to be a way to make
SchemaQuery queries version-dependent.

Thanks for the review Tom.

I was avoiding changing things that already worked and hence ended up
with the ad-hoc code. It's much better have a unified approach to
handling this, though.

So ... here is a modest proposal. It invents a VersionedQuery concept
and also extends the SchemaQuery infrastructure to allow those to be
versioned. I have not taken this nearly as far as it could be taken,
since it's mostly just proposing mechanism. To illustrate the
VersionedQuery infrastructure, I fixed it so it wouldn't send
publication/subscription queries to pre-v10 servers, and to illustrate
the versioned SchemaQuery infrastructure, I fixed the prokind problems.

(Hmm, I guess the new prokind column may be germane to my query for
callable functions.)

If people like this approach, I propose to commit this more or less
as-is. The select-tab-completion patch would then need to be rewritten
to use this infrastructure, but I think that should be straightforward.

The SELECT completion patch was definitely ugly. I think the
VersionedQuery is a nice way of making it less ad-hoc, but the work of
writing the numerous query versions, where necessary, remains.

My patch had 4 versions: PRE_81, PRE_90, PRE_96, and current. This
could be reformulated as

static const VersionedQuery
Query_for_list_of_selectable_functions_or_attributes[] = {
{90600, ... },
{90000, ... },
{80100, ... },
{70300, ... },
{0, NULL}
};

I do think the version indicators are nicer when they mean "supported
as of this server version" rather than my "fallback if the server
version is prior to this".

Is it overkill to have so many variations?

I am still just slightly unclear on where we are in relation to the
SAVEPOINT patch -- is that redundant now?

For SELECT support, I would be happy with applying:

a. Your patch
b. A reworked simple SELECT patch (possibly with fewer query versions)
c. A SAVEPOINT patch *if* it's deemed useful [1]There is more complexity with tab completions and transactions than I want to tackle just for SELECT; see /messages/by-id/CAMyN-kAyFTC4Xavp+D6XYOoAOZQW2=c79htji06DXF+uF6StOQ@mail.gmail.com

And perhaps later,
d. A cleverer SELECT patch (supporting additional items after the
first comma, for instance)

As a separate line of work, the infrastructure could be applied to fix
the pre-existing places where tab completion fails against old servers.
But that's probably work for v12 or beyond, unless somebody's really
motivated to do it right now.

regards, tom lane

Edmund

[1]: There is more complexity with tab completions and transactions than I want to tackle just for SELECT; see /messages/by-id/CAMyN-kAyFTC4Xavp+D6XYOoAOZQW2=c79htji06DXF+uF6StOQ@mail.gmail.com
than I want to tackle just for SELECT; see
/messages/by-id/CAMyN-kAyFTC4Xavp+D6XYOoAOZQW2=c79htji06DXF+uF6StOQ@mail.gmail.com

#26Edmund Horner
ejrh00@gmail.com
In reply to: Vik Fearing (#24)
Re: PATCH: psql tab completion for SELECT

On 5 March 2018 at 21:46, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

Tab completion on functions in SELECT (a subset of this thread's patch)
is quite important to me personally. I will work on this in the coming
days.

It's something I've missed numerous times in the last few months at
work. I guess I should really be running a psql with my own
preliminary patches applied; but I'm only a novice pgsql-hacker, and
easily default to using the official one.

If you are going to work on a patch just for functions, you should
probably make it a SchemaQuery. I did not find a way to support
schema-qualified functions in a query that also returned column names.

Cheers,
Edmund

#27Tom Lane
tgl@sss.pgh.pa.us
In reply to: Edmund Horner (#25)
Re: PATCH: psql tab completion for SELECT

Edmund Horner <ejrh00@gmail.com> writes:

On 5 March 2018 at 08:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:

If people like this approach, I propose to commit this more or less
as-is. The select-tab-completion patch would then need to be rewritten
to use this infrastructure, but I think that should be straightforward.

My patch had 4 versions: PRE_81, PRE_90, PRE_96, and current. This
could be reformulated as
static const VersionedQuery
Query_for_list_of_selectable_functions_or_attributes[] = {
{90600, ... },
{90000, ... },
{80100, ... },
{70300, ... },
{0, NULL}
};

Right.

Is it overkill to have so many variations?

Well, it's whatever you need for the purpose. We could discuss what the
support cutoff is, but I doubt we'd make it any later than 8.0, so some
types of catalog queries are going to need a lot of variations.

I am still just slightly unclear on where we are in relation to the
SAVEPOINT patch -- is that redundant now?

I'm inclined to think it's a bit pointless, if the direction we're
heading is to make the queries actually work on every server version.
Issuing a savepoint would just mask failures.

What would be actually useful is to be able to tab-complete even in
the midst of a failed transaction block ... but savepoints as such
won't get us there, and I have no good ideas about what would.

regards, tom lane

#28David G. Johnston
david.g.johnston@gmail.com
In reply to: Tom Lane (#27)
Re: PATCH: psql tab completion for SELECT

On Mon, Mar 5, 2018 at 7:41 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

What would be actually useful is to be able to tab-complete even in
the midst of a failed transaction block ... but savepoints as such
won't get us there, and I have no good ideas about what would.

​Why not have psql open two sessions to the backend, one with
application_name 'psql_user' and one with application name "psql_​meta" (or
some such) and have all these queries executed on the psql_meta connection?

David J.

#29Tom Lane
tgl@sss.pgh.pa.us
In reply to: David G. Johnston (#28)
Re: PATCH: psql tab completion for SELECT

"David G. Johnston" <david.g.johnston@gmail.com> writes:

On Mon, Mar 5, 2018 at 7:41 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

What would be actually useful is to be able to tab-complete even in
the midst of a failed transaction block ... but savepoints as such
won't get us there, and I have no good ideas about what would.

​Why not have psql open two sessions to the backend, one with
application_name 'psql_user' and one with application name "psql_​meta" (or
some such) and have all these queries executed on the psql_meta connection?

If we did it like that, tab completion would fail to see the session's
temp tables, or objects created in the current open transaction.

People might bitch about using twice as many connections, too, although
likely you could finesse that by only opening the second connection if
tab completion actually happens (so that only interactive sessions have
one). Still, the local-objects problem seems like a fatal objection.

regards, tom lane

#30Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#26)
Re: PATCH: psql tab completion for SELECT

On 03/05/2018 11:33 AM, Edmund Horner wrote:

On 5 March 2018 at 21:46, Vik Fearing <vik.fearing@2ndquadrant.com> wrote:

Tab completion on functions in SELECT (a subset of this thread's patch)
is quite important to me personally. I will work on this in the coming
days.

It's something I've missed numerous times in the last few months at
work. I guess I should really be running a psql with my own
preliminary patches applied; but I'm only a novice pgsql-hacker, and
easily default to using the official one.

If you are going to work on a patch just for functions, you should
probably make it a SchemaQuery. I did not find a way to support
schema-qualified functions in a query that also returned column names.

I meant that I was going to work on your patch, with you, to quickly get
this in v11.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#31Vik Fearing
vik.fearing@2ndquadrant.com
In reply to: Edmund Horner (#25)
Re: PATCH: psql tab completion for SELECT

On 03/05/2018 11:21 AM, Edmund Horner wrote:

I am still just slightly unclear on where we are in relation to the
SAVEPOINT patch -- is that redundant now?

My vote is to reject that patch.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

#32Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#23)
Re: PATCH: psql tab completion for SELECT

I wrote:

If people like this approach, I propose to commit this more or less
as-is. The select-tab-completion patch would then need to be rewritten
to use this infrastructure, but I think that should be straightforward.
As a separate line of work, the infrastructure could be applied to fix
the pre-existing places where tab completion fails against old servers.
But that's probably work for v12 or beyond, unless somebody's really
motivated to do it right now.

Pushed, but while looking it over a second time, I noticed a discrepancy
that ought to be resolved. In Query_for_list_of_functions, we now have
this for server version >= 11

/* selcondition */
"p.prokind IN ('f', 'w')",

and this for prior versions:

/* selcondition */
NULL,

The prokind variant is as Peter had it, and NULL is what we were using
here in v10 and earlier. But it strikes me that these are inconsistent,
in that the prokind variant rejects aggregates while the other variant
doesn't. We should decide which behavior we want and make that happen
consistently regardless of server version.

I believe the primary reason why the old code didn't reject aggregates
is that there is no GRANT ON AGGREGATE syntax, so that we really need to
include aggregates when completing GRANT ... ON FUNCTION. Also, other
commands such as DROP FUNCTION will accept an aggregate, although that's
arguably just historical laxity.

If we wanted to tighten this up, one way would be to create a separate
Query_for_list_of_functions_or_aggregates that allows both, and use it
for (only) the GRANT/REVOKE case. I'm not sure it's worth the trouble
though; I do not recall hearing field complaints about the laxity of
the existing completion rules. So my inclination is to change the
v11 code to be "p.prokind != 'p'" and leave it at that.

Thoughts?

regards, tom lane

#33Edmund Horner
ejrh00@gmail.com
In reply to: Tom Lane (#27)
1 attachment(s)
Re: PATCH: psql tab completion for SELECT

I've reworked the SELECT completion patch to use the VersionedQuery
infrastructure.

I've also made it a schema query (for the functions), with an addon
for the attributes. This provides schema-aware completion.

Previously, addons to schema queries were appended verbatim; I've
changed this to use the same interpolation just as in the simple_query
case, so that the attributes can be filtered in the query. Otherwise,
relevant items can be omitted when they don't make it into the first
1000 rows in the query result, even when only a small number of items
are ultimately presented for completion.

Edmund

Attachments:

psql-select-tab-completion-v5.patchapplication/octet-stream; name=psql-select-tab-completion-v5.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 340febe..8504cd9 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1170,6 +1170,109 @@ static const VersionedQuery Query_for_list_of_subscriptions[] = {
 };
 
 /*
+ * We provide multiple versions of this query, so that some useful
+ * functionality is present even for older versions.
+ */
+static const SchemaQuery Query_for_list_of_selectable_functions[] = {
+	{
+		/* min_server_version */
+		90600,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		"AND 'internal'::regtype <> ALL (p.proargtypes) "
+		"AND p.oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "
+		"AND p.oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn,aggcombinefn,aggserialfn,aggdeserialfn,aggmtransfn,aggminvtransfn,aggmfinalfn]) FROM pg_aggregate) "
+		"AND p.oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "
+		"AND p.oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "
+		"AND p.oid NOT IN (SELECT castfunc FROM pg_cast) "
+		"AND p.oid NOT IN (SELECT amproc FROM pg_amproc) ",
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{
+		/* min_server_version */
+		90000,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		"AND 'internal'::regtype != ALL (p.proargtypes) "
+		"AND p.oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "
+		"AND p.oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn]) FROM pg_aggregate) "
+		"AND p.oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "
+		"AND p.oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "
+		"AND p.oid NOT IN (SELECT castfunc FROM pg_cast) "
+		"AND p.oid NOT IN (SELECT amproc FROM pg_amproc) "
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{
+		/* min_server_version */
+		80100,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		"'internal'::regtype != ALL ([.proargtypes) "
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{
+		/* min_server_version */
+		70400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{0, NULL}
+};
+
+/*
+ * This addon is used to find (unqualified) column names to include
+ * alongside the function names from the query above.
+ */
+static const VersionedQuery Query_addon_for_list_of_selectable_attributes[] = {
+	{70400,
+		"    UNION ALL "
+		"   SELECT DISTINCT pg_catalog.quote_ident(attname) "
+		"     FROM pg_catalog.pg_attribute "
+		"    WHERE attnum > 0 "
+		"      AND NOT attisdropped "
+		"      AND substring(pg_catalog.quote_ident(attname),1,%d)='%s'"
+	},
+	{0, NULL}
+};
+
+/*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
  */
@@ -3396,7 +3499,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (TailMatches1("SELECT") || TailMatches2("SELECT", "ALL|DISTINCT"))
+		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
+						     Query_addon_for_list_of_selectable_attributes);
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
@@ -4003,7 +4108,8 @@ complete_from_versioned_schema_query(const char *text, int state)
  * where %d is the string length of the text and %s the text itself.
  *
  * If both simple_query and schema_query are non-NULL, then we construct
- * a schema query and append the (uninterpreted) string simple_query to it.
+ * a schema query and append the simple_query to it, replacing the %d and %s
+ * as described above.
  *
  * It is assumed that strings should be escaped to become SQL literals
  * (that is, what is in the query is actually ... '%s' ...)
@@ -4150,21 +4256,18 @@ _complete_from_query(const char *simple_query,
 							  " WHERE substring(pg_catalog.quote_ident(nspname) || '.',1,%d) ="
 							  " substring('%s',1,pg_catalog.length(pg_catalog.quote_ident(nspname))+1)) = 1",
 							  char_length, e_text);
-
-			/* If an addon query was provided, use it */
-			if (simple_query)
-				appendPQExpBuffer(&query_buffer, "\n%s", simple_query);
 		}
 		else
 		{
 			Assert(simple_query);
-			/* simple_query is an sprintf-style format string */
-			appendPQExpBuffer(&query_buffer, simple_query,
-							  char_length, e_text,
-							  e_info_charp, e_info_charp,
-							  e_info_charp2, e_info_charp2);
 		}
 
+		/* simple_query is an sprintf-style format string. */
+		appendPQExpBuffer(&query_buffer, simple_query,
+						  char_length, e_text,
+						  e_info_charp, e_info_charp,
+						  e_info_charp2, e_info_charp2);
+
 		/* Limit the number of records in the result */
 		appendPQExpBuffer(&query_buffer, "\nLIMIT %d",
 						  completion_max_records);
#34Edmund Horner
ejrh00@gmail.com
In reply to: Edmund Horner (#33)
1 attachment(s)
Re: PATCH: psql tab completion for SELECT

New patch that fixes a little bug with appending NULL addons to schema queries.

Attachments:

psql-select-tab-completion-v6.patchapplication/octet-stream; name=psql-select-tab-completion-v6.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 340febe..4144562 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1170,6 +1170,109 @@ static const VersionedQuery Query_for_list_of_subscriptions[] = {
 };
 
 /*
+ * We provide multiple versions of this query, so that some useful
+ * functionality is present even for older versions.
+ */
+static const SchemaQuery Query_for_list_of_selectable_functions[] = {
+	{
+		/* min_server_version */
+		90600,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		"AND 'internal'::regtype <> ALL (p.proargtypes) "
+		"AND p.oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "
+		"AND p.oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn,aggcombinefn,aggserialfn,aggdeserialfn,aggmtransfn,aggminvtransfn,aggmfinalfn]) FROM pg_aggregate) "
+		"AND p.oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "
+		"AND p.oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "
+		"AND p.oid NOT IN (SELECT castfunc FROM pg_cast) "
+		"AND p.oid NOT IN (SELECT amproc FROM pg_amproc) ",
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{
+		/* min_server_version */
+		90000,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		"AND 'internal'::regtype != ALL (p.proargtypes) "
+		"AND p.oid NOT IN (SELECT unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze]) FROM pg_type) "
+		"AND p.oid NOT IN (SELECT unnest(array[aggtransfn,aggfinalfn]) FROM pg_aggregate) "
+		"AND p.oid NOT IN (SELECT unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "
+		"AND p.oid NOT IN (SELECT unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "
+		"AND p.oid NOT IN (SELECT castfunc FROM pg_cast) "
+		"AND p.oid NOT IN (SELECT amproc FROM pg_amproc) "
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{
+		/* min_server_version */
+		80100,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		"'internal'::regtype != ALL ([.proargtypes) "
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{
+		/* min_server_version */
+		70400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{0, NULL}
+};
+
+/*
+ * This addon is used to find (unqualified) column names to include
+ * alongside the function names from the query above.
+ */
+static const VersionedQuery Query_addon_for_list_of_selectable_attributes[] = {
+	{70400,
+		"    UNION ALL "
+		"   SELECT DISTINCT pg_catalog.quote_ident(attname) "
+		"     FROM pg_catalog.pg_attribute "
+		"    WHERE attnum > 0 "
+		"      AND NOT attisdropped "
+		"      AND substring(pg_catalog.quote_ident(attname),1,%d)='%s'"
+	},
+	{0, NULL}
+};
+
+/*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
  */
@@ -3396,7 +3499,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (TailMatches1("SELECT") || TailMatches2("SELECT", "ALL|DISTINCT"))
+		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
+							Query_addon_for_list_of_selectable_attributes);
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
@@ -4003,7 +4108,8 @@ complete_from_versioned_schema_query(const char *text, int state)
  * where %d is the string length of the text and %s the text itself.
  *
  * If both simple_query and schema_query are non-NULL, then we construct
- * a schema query and append the (uninterpreted) string simple_query to it.
+ * a schema query and append the simple_query to it, replacing the %d and %s
+ * as described above.
  *
  * It is assumed that strings should be escaped to become SQL literals
  * (that is, what is in the query is actually ... '%s' ...)
@@ -4150,21 +4256,22 @@ _complete_from_query(const char *simple_query,
 							  " WHERE substring(pg_catalog.quote_ident(nspname) || '.',1,%d) ="
 							  " substring('%s',1,pg_catalog.length(pg_catalog.quote_ident(nspname))+1)) = 1",
 							  char_length, e_text);
-
-			/* If an addon query was provided, use it */
-			if (simple_query)
-				appendPQExpBuffer(&query_buffer, "\n%s", simple_query);
 		}
 		else
 		{
 			Assert(simple_query);
-			/* simple_query is an sprintf-style format string */
-			appendPQExpBuffer(&query_buffer, simple_query,
-							  char_length, e_text,
-							  e_info_charp, e_info_charp,
-							  e_info_charp2, e_info_charp2);
 		}
 
+		/*
+		 * simple_query is an sprintf-style format string (or it could be NULL, but
+		 * only if this is a schema query with no addon).
+		 */
+		if (simple_query)
+			appendPQExpBuffer(&query_buffer, simple_query,
+					  char_length, e_text,
+					  e_info_charp, e_info_charp,
+					  e_info_charp2, e_info_charp2);
+
 		/* Limit the number of records in the result */
 		appendPQExpBuffer(&query_buffer, "\nLIMIT %d",
 						  completion_max_records);
#35Edmund Horner
ejrh00@gmail.com
In reply to: Edmund Horner (#34)
Re: PATCH: psql tab completion for SELECT

Some updates on patch status:
- "version-dependent-tab-completion-1.patch" by Tom Lane was committed in 722408bcd.
- "psql-tab-completion-savepoint-v1.patch" by Edmund Horner is probably not needed.
- "psql-select-tab-completion-v6.patch" (the latest) is still under development/review.

#36Edmund Horner
ejrh00@gmail.com
In reply to: Edmund Horner (#34)
Re: PATCH: psql tab completion for SELECT

On 8 March 2018 at 17:23, Edmund Horner <ejrh00@gmail.com> wrote:

New patch that fixes a little bug with appending NULL addons to schema queries.

Hi all, I haven't heard anything for a while and so assume you're
beavering away on real features. :)

I've been dogfooding this patch at work, and I am personally pretty
happy with it.

I still think the number of completions on an empty string is a bit
too big, but I don't know what to omit. There are around 1700
completions on the empty "postgres" database in my testing, and we
show the first 1000 (arbitrarily limited, as the generated SQL has
LIMIT 1000 but no ORDER BY).

Should we just leave it as is?

Whether we improve the filtering or not, I'm optimistic the feature
will be committed in this CF or the next.

I've also been working on adding support for completions after commas,
but I really want to see the current feature finished first.

Cheers,
Edmund

#37Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Edmund Horner (#36)
Re: PATCH: psql tab completion for SELECT

On 3/21/18 02:51, Edmund Horner wrote:

I still think the number of completions on an empty string is a bit
too big, but I don't know what to omit. There are around 1700
completions on the empty "postgres" database in my testing, and we
show the first 1000 (arbitrarily limited, as the generated SQL has
LIMIT 1000 but no ORDER BY).

Should we just leave it as is?

Whether we improve the filtering or not, I'm optimistic the feature
will be committed in this CF or the next.

I looked at this a bit now. I think it still needs some work.

Some of the queries for older versions contain syntax errors that causes
them not to work.

For example, in 80100:

"'internal'::regtype != ALL ([.proargtypes) "

The query definition structures are missing a comma between selcondition
and viscondition. This causes all the following fields to be
misassigned. I'm not quite sure how it actually works at all some of
the time. There might be another bug that offsets this one.

I'd like to see a short comment what is different between each of the
version queries. You had a list earlier in the thread.

The selection of which functions to show can be further refined.

I don't think the contents of pg_amproc and pg_cast should be excluded.
Some of those functions are useful. Similarly for pg_operator.oprcode.

Things like oprrest and oprjoin will already be excluded because they
have "internal" arguments. Similarly for some columns in pg_aggregate.

There are also additional pseudo-types that should be excluded. See
pg_type.typtype = 'p', except some of those should not be excluded.
Needs more thought.

Considering these issues, I think it would be appropriate to move this
patch to the next commitfest.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#38Edmund Horner
ejrh00@gmail.com
In reply to: Peter Eisentraut (#37)
Re: PATCH: psql tab completion for SELECT

On 6 April 2018 at 13:29, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:

I looked at this a bit now. I think it still needs some work.

Hi Peter, thanks for the feedback.

Some of the queries for older versions contain syntax errors that causes
them not to work.

For example, in 80100:

"'internal'::regtype != ALL ([.proargtypes) "

The query definition structures are missing a comma between selcondition
and viscondition. This causes all the following fields to be
misassigned. I'm not quite sure how it actually works at all some of
the time. There might be another bug that offsets this one.

One of the problems was a missing comma before the viscondition value
in the struct!

/* selcondition */
"p.prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) "
...
"AND p.oid NOT IN (SELECT amproc FROM pg_amproc) " <--
Should be a comma here!
/* viscondition */
"pg_catalog.pg_function_is_visible(p.oid)",

I did some fairly thorough testing against older servers, but that was
before I rewrote it to fit in Tom's versioned SchemaQuery. I'll fix
this and repeat the testing.

I'd like to see a short comment what is different between each of the
version queries. You had a list earlier in the thread.

Ok, I'll see if I can add that.

The selection of which functions to show can be further refined.

I don't think the contents of pg_amproc and pg_cast should be excluded.
Some of those functions are useful. Similarly for pg_operator.oprcode.

Things like oprrest and oprjoin will already be excluded because they
have "internal" arguments. Similarly for some columns in pg_aggregate.

There are also additional pseudo-types that should be excluded. See
pg_type.typtype = 'p', except some of those should not be excluded.
Needs more thought.

Ok I'll play with these.

Selection of which functions and attributes to show is something with
probably no right answer, so we might have to settle for "close
enough" at some point.

Show quoted text

Considering these issues, I think it would be appropriate to move this
patch to the next commitfest.

#39Edmund Horner
ejrh00@gmail.com
In reply to: Peter Eisentraut (#37)
Re: PATCH: psql tab completion for SELECT

On 6 April 2018 at 13:29, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:

The selection of which functions to show can be further refined.

I don't think the contents of pg_amproc and pg_cast should be excluded.
Some of those functions are useful. Similarly for pg_operator.oprcode.

Things like oprrest and oprjoin will already be excluded because they
have "internal" arguments. Similarly for some columns in pg_aggregate.

You're right. There were lots of useful functions being excluded by
the pg_amproc, pg_cast, and oprcode checks. And all the oprrest and
oprjoin functions are already excluded, so I can remove that check.

Perhaps we should remove the "appears in this catalog table" exclusion
checks, and just use argument and return type?

There are also additional pseudo-types that should be excluded. See
pg_type.typtype = 'p', except some of those should not be excluded.
Needs more thought.

I don't know much about some of the pseudotypes but this is what I propose:

any*, record, _record, cstring should NOT be excluded
void should NOT be excluded for return type (and perhaps in general;
void_out and void_send are callable, if not hugely useful in psql)

trigger, event_trigger should be excluded
internal, opaque, unknown, pg_ddl_command should be excluded
language_handler, tsm_handler, index_am_handler, fdw_handler should be excluded

For modern servers, our query can be simplified to something like:

SELECT distinct pg_catalog.quote_ident(p.proname)
FROM pg_catalog.pg_proc p
WHERE NOT arrayoverlap(ARRAY['internal', 'event_trigger', 'internal',
'opaque', 'unknown', 'pg_ddl_command', 'language_handler',
'tsm_handler', 'index_am_handler', 'fdw_handler']::regtype[]::oid[],
ARRAY(SELECT p.prorettype UNION SELECT unnest(proargtypes)));

What do you think?

#40Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Edmund Horner (#36)
Re: PATCH: psql tab completion for SELECT

(trimmed CC list to evade gmail's spam filter, sorry)

On 21/03/18 08:51, Edmund Horner wrote:

Hi all, I haven't heard anything for a while and so assume you're
beavering away on real features. :)

I've been dogfooding this patch at work, and I am personally pretty
happy with it.

I still think the number of completions on an empty string is a bit
too big, but I don't know what to omit. There are around 1700
completions on the empty "postgres" database in my testing, and we
show the first 1000 (arbitrarily limited, as the generated SQL has
LIMIT 1000 but no ORDER BY).

Should we just leave it as is?

Whether we improve the filtering or not, I'm optimistic the feature
will be committed in this CF or the next.

I've also been working on adding support for completions after commas,
but I really want to see the current feature finished first.

Playing around with this a little bit, I'm not very satisfied with the
completions. Sometimes this completes too much, and sometimes too
little. All of this has been mentioned in this and the other thread [1]/messages/by-id/1328820579.11241.4.camel@vanquo.pezone.net
already, this just my opinion on all this.

Too much:

postgres=# \d lp
Table "public.lp"
Column | Type | Collation | Nullable | Default
----------+---------+-----------+----------+---------
id | integer | | |
partkey | text | | |
partcol1 | text | | |
partcol2 | text | | |
Partition key: LIST (partkey)
Number of partitions: 1000 (Use \d+ to list them.)

postgres=# select pa[TAB]
pad_attribute parameter_default parameter_style partattrs
partcol2 partexprs partrelid
page parameter_mode parameter_types partclass
partcollation partkey partstrat
pageno parameter_name parse_ident( partcol1
partdefid partnatts passwd

Too little:

postgres=# select partkey, p[TAB]
[no completions]

Then there's the multi-column case, which seems weird (to a user - I
know the implementation and understand why):

postgres=# select partkey, partc[TAB]
[no completions]

And I'd love this case, where go back to edit the SELECT list, after
already typing the FROM part, to be smarter:

postgres=# select p[TAB] FROM lp;
Display all 370 possibilities? (y or n)

There's something weird going on with system columns, from a user's
point of view:

SELECT oi[TAB]
oid oidvectortypes(

postgres=# select xm[TAB]
xmin xmlcomment( xml_is_well_formed_content(
xmlvalidate(
xmlagg( xml_is_well_formed(
xml_is_well_formed_document(

So oid and xmin are completed. But "xmax" and "ctid" are not. I think
this is because in fact none of the system columns are completed, but
there happen to be some tables with regular columns named "oid" and
"xmin". So it makes sense once you know that, but it's pretty confusing
to a user. Tab-completion is a great way for a user to explore what's
available, so it's weird that some system columns are effectively
completed while others are not.

I'd like to not include set-returning functions from the list. Although
you can do "SELECT generate_series(1,10)", I'd like to discourage people
from doing that, since using set-returning functions in the target list
is a PostgreSQL extension to the SQL standard, and IMHO the "SELECT *
FROM generate_series(1,10)" syntax is easier to understand and works
more sanely with joins etc. Conversely, it would be really nice to
include set-returning function in the completions after FROM.

I understand that there isn't much you can do about some of those
things, short of adding a ton of new context information about previous
queries and heuristics. I think the completion needs to be smarter to be
acceptable. I don't know what exactly to do, but perhaps someone else
has ideas.

I'm also worried about performance, especially of the query to get all
the column names. It's pretty much guaranteed to do perform a
SeqScan+Sort+Unique on pg_attribute. pg_attribute can be very large. In
my little test database, with the above 1000-partition table, hitting
tab after "SELECT " takes about 1 second, until you get the "Display all
1000 possibilities" prompt. And that's not a particularly large schema.

- Heikki

PS. All the references to "pg_attribute" and other system tables, and
functions, need to be schema-qualified, as "pg_catalog.pg_attribute" and
so forth.

[1]: /messages/by-id/1328820579.11241.4.camel@vanquo.pezone.net
/messages/by-id/1328820579.11241.4.camel@vanquo.pezone.net

#41Joao De Almeida Pereira
jdealmeidapereira@pivotal.io
In reply to: Heikki Linnakangas (#40)
Re: PATCH: psql tab completion for SELECT

Hello,

postgres=# select partkey, partc[TAB]

[no completions]

From the thread, I believe that this feature will be implemented in a after
patch.

And I'd love this case, where go back to edit the SELECT list, after
already typing the FROM part, to be smarter:

postgres=# select p[TAB] FROM lp;
Display all 370 possibilities? (y or n)

I believe this would be a very interesting feature indeed.

After playing alittle bit around with the patch I noticed that a comma was
missing in line 1214
+ 1202                 /* min_server_version */
+ 1203                 90000,
+ 1204                 /* catname */
+ 1205                 "pg_catalog.pg_proc p",
+ 1206                 /* selcondition */
+ 1207                 "p.prorettype NOT IN ('trigger'::regtype,
'internal'::regtype) "
+ 1208                 "AND 'internal'::regtype != ALL (p.proargtypes) "
+ 1209                 "AND p.oid NOT IN (SELECT
unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze])
FROM pg_type) "
+ 1210                 "AND p.oid NOT IN (SELECT
unnest(array[aggtransfn,aggfinalfn]) FROM pg_aggregate) "
+ 1211                 "AND p.oid NOT IN (SELECT
unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "
+ 1212                 "AND p.oid NOT IN (SELECT
unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "
+ 1213                 "AND p.oid NOT IN (SELECT castfunc FROM pg_cast) "
+ 1214                 "AND p.oid NOT IN (SELECT amproc FROM pg_amproc) "
+ 1215                 /* viscondition */
+ 1216                 "pg_catalog.pg_function_is_visible(p.oid)",

To catch these typos would be good if we could get some testing around
psql.
(Newbie question: do we have any kind of testing around tools like psql?)

Thanks
Joao

#42Edmund Horner
ejrh00@gmail.com
In reply to: Heikki Linnakangas (#40)
2 attachment(s)
Re: PATCH: psql tab completion for SELECT

On 17 July 2018 at 00:00, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

Playing around with this a little bit, I'm not very satisfied with the
completions. Sometimes this completes too much, and sometimes too little.
All of this has been mentioned in this and the other thread [1] already,
this just my opinion on all this.

Hi Heikki, thanks for getting this thread active again.

I do actually have another patch, which I didn't post yet because
everyone was busy with v11, and I wanted to wait for more feedback
about inclusions/exclusions. Holding it back now seems like a mistake
(especially since the last patch had a bug) so here it is.

There's also a patch, to be applied on top, that adds completion after commas.

Too much:

postgres=# \d lp
Table "public.lp"
Column | Type | Collation | Nullable | Default
----------+---------+-----------+----------+---------
id | integer | | |
partkey | text | | |
partcol1 | text | | |
partcol2 | text | | |
Partition key: LIST (partkey)
Number of partitions: 1000 (Use \d+ to list them.)

postgres=# select pa[TAB]
pad_attribute parameter_default parameter_style partattrs partcol2
partexprs partrelid
page parameter_mode parameter_types partclass
partcollation partkey partstrat
pageno parameter_name parse_ident( partcol1 partdefid
partnatts passwd

I agree that there is too much. I don't know what the best way to
reduce it is, short of specifically excluding certain things. In
theory, I think the query could be sensitive to how much text is
entered, for example, let pa[TAB] not show partrelid and other columns
from pg_partition, but part[TAB] show them; the general rule could be
"only show attributes for system tables if 3 or more letters entered".
But I'm wary of overcomplicating a patch that is (IMHO) a useful
improvement even if simple.

Too little:

postgres=# select partkey, p[TAB]
[no completions]

Then there's the multi-column case, which seems weird (to a user - I know
the implementation and understand why):

postgres=# select partkey, partc[TAB]
[no completions]

Yep, there was no completion after commas in the previous patches.

And I'd love this case, where go back to edit the SELECT list, after already
typing the FROM part, to be smarter:

postgres=# select p[TAB] FROM lp;
Display all 370 possibilities? (y or n)

I don't know enough about readline to estimate how easy this would be.
I think all our current completions use only the text up to the
cursor.

There's something weird going on with system columns, from a user's point of
view:

SELECT oi[TAB]
oid oidvectortypes(

postgres=# select xm[TAB]
xmin xmlcomment( xml_is_well_formed_content(
xmlvalidate(
xmlagg( xml_is_well_formed(
xml_is_well_formed_document(

So oid and xmin are completed. But "xmax" and "ctid" are not. I think this
is because in fact none of the system columns are completed, but there
happen to be some tables with regular columns named "oid" and "xmin". So it
makes sense once you know that, but it's pretty confusing to a user.
Tab-completion is a great way for a user to explore what's available, so
it's weird that some system columns are effectively completed while others
are not.

You are correct, system columns are excluded, but of course there are
always the same column names in other tables. Do you think we should
just include all columns?

I'd like to not include set-returning functions from the list. Although you
can do "SELECT generate_series(1,10)", I'd like to discourage people from
doing that, since using set-returning functions in the target list is a
PostgreSQL extension to the SQL standard, and IMHO the "SELECT * FROM
generate_series(1,10)" syntax is easier to understand and works more sanely
with joins etc. Conversely, it would be really nice to include set-returning
function in the completions after FROM.

I'm happy to exclude SRFs after SELECT if others agree. Including
them after FROM seems worthwhile, but best left for a different patch?

I understand that there isn't much you can do about some of those things,
short of adding a ton of new context information about previous queries and
heuristics. I think the completion needs to be smarter to be acceptable. I
don't know what exactly to do, but perhaps someone else has ideas.

I'm also worried about performance, especially of the query to get all the
column names. It's pretty much guaranteed to do perform a
SeqScan+Sort+Unique on pg_attribute. pg_attribute can be very large. In my
little test database, with the above 1000-partition table, hitting tab after
"SELECT " takes about 1 second, until you get the "Display all 1000
possibilities" prompt. And that's not a particularly large schema.

The indexes on pg_attribute both start with attrelid; unless we know
what small set of relations we want attributes for, I don't think we
can avoid a SeqScan. Maybe there's a way to extract the attribute
name part from the text entered and use it in a "WHERE attname LIKE
'%s' " qualifier, to encourage an IndexOnlyScan for some cases.

PS. All the references to "pg_attribute" and other system tables, and
functions, need to be schema-qualified, as "pg_catalog.pg_attribute" and so
forth.

(Thanks for the advice. I managed to sneak this into the updated patch.)

Attachments:

psql-select-tab-completion-v7.patchapplication/octet-stream; name=psql-select-tab-completion-v7.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index bb696f8..90516a1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1170,6 +1170,63 @@ static const VersionedQuery Query_for_list_of_subscriptions[] = {
 	{0, NULL}
 };
 
+static const SchemaQuery Query_for_list_of_selectable_functions[] = {
+	/* For servers since 8.4, which introduced the unnest function. */
+	{
+		/* min_server_version */
+		80400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"NOT pg_catalog.arrayoverlap( "
+		"	ARRAY(SELECT oid FROM pg_catalog.pg_type WHERE typnamespace IN (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = 'pg_catalog') "
+		"		AND typname = ANY (ARRAY['internal', 'event_trigger', 'internal','opaque', 'unknown', 'pg_ddl_command', 'language_handler','tsm_handler', 'index_am_handler', 'fdw_handler'])), "
+		"	ARRAY(SELECT p.prorettype UNION SELECT pg_catalog.unnest(proargtypes)))",
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	/* For older servers since 7.4. */
+	{
+		/* min_server_version */
+		70400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) ",
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{0, NULL}
+};
+
+/*
+ * This addon is used to find (unqualified) column names to include
+ * alongside the function names from the query above.
+ */
+static const VersionedQuery Query_addon_for_list_of_selectable_attributes[] = {
+	{70400,
+		"    UNION ALL "
+		"   SELECT DISTINCT pg_catalog.quote_ident(attname) "
+		"     FROM pg_catalog.pg_attribute "
+		"    WHERE attnum > 0 "
+		"      AND NOT attisdropped "
+		"      AND substring(pg_catalog.quote_ident(attname),1,%d)='%s'"
+	},
+	{0, NULL}
+};
+
 /*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
@@ -3400,7 +3457,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (TailMatches1("SELECT") || TailMatches2("SELECT", "ALL|DISTINCT"))
+		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
+							Query_addon_for_list_of_selectable_attributes);
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
@@ -4011,7 +4070,8 @@ complete_from_versioned_schema_query(const char *text, int state)
  * where %d is the string length of the text and %s the text itself.
  *
  * If both simple_query and schema_query are non-NULL, then we construct
- * a schema query and append the (uninterpreted) string simple_query to it.
+ * a schema query and append the simple_query to it, replacing the %d and %s
+ * as described above.
  *
  * It is assumed that strings should be escaped to become SQL literals
  * (that is, what is in the query is actually ... '%s' ...)
@@ -4158,21 +4218,22 @@ _complete_from_query(const char *simple_query,
 							  " WHERE substring(pg_catalog.quote_ident(nspname) || '.',1,%d) ="
 							  " substring('%s',1,pg_catalog.length(pg_catalog.quote_ident(nspname))+1)) = 1",
 							  char_length, e_text);
-
-			/* If an addon query was provided, use it */
-			if (simple_query)
-				appendPQExpBuffer(&query_buffer, "\n%s", simple_query);
 		}
 		else
 		{
 			Assert(simple_query);
-			/* simple_query is an sprintf-style format string */
-			appendPQExpBuffer(&query_buffer, simple_query,
-							  char_length, e_text,
-							  e_info_charp, e_info_charp,
-							  e_info_charp2, e_info_charp2);
 		}
 
+		/*
+		 * simple_query is an sprintf-style format string (or it could be NULL, but
+		 * only if this is a schema query with no addon).
+		 */
+		if (simple_query)
+			appendPQExpBuffer(&query_buffer, simple_query,
+					  char_length, e_text,
+					  e_info_charp, e_info_charp,
+					  e_info_charp2, e_info_charp2);
+
 		/* Limit the number of records in the result */
 		appendPQExpBuffer(&query_buffer, "\nLIMIT %d",
 						  completion_max_records);
select-comma-completion-v1.patchapplication/octet-stream; name=select-comma-completion-v1.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 90516a1..58e139b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1473,6 +1473,35 @@ ends_with(const char *s, char c)
 }
 
 /*
+ * Get the last keyword matching a pattern.
+ */
+static const char *
+last_keyword(char **previous_words, int previous_words_count, const char *keyword_pattern) {
+	int i;
+
+	for (i = 0; i < previous_words_count; i++) {
+		if (word_matches(keyword_pattern, previous_words[i])) {
+			return previous_words[i];
+		}
+	}
+
+	return NULL;
+}
+
+/*
+ * Was the last keyword one of the expected ones?
+ * E.g. last_keyword("SELECT|FROM|WHERE|GROUP|ORDER", "SELECT") is true if SELECT is the most recent
+ * of those key words to appear.
+ */
+static bool
+last_keyword_matches(char **previous_words, int previous_words_count, const char *keyword_pattern, const char *accepted_pattern) {
+	const char *last_kw = last_keyword(previous_words, previous_words_count, keyword_pattern);
+	if (!last_kw)
+		return false;
+	return word_matches(last_kw, accepted_pattern);
+}
+
+/*
  * The completion function.
  *
  * According to readline spec this gets passed the text entered so far and its
@@ -1512,6 +1541,14 @@ psql_completion(const char *text, int start, int end)
 #define prev8_wd  (previous_words[7])
 #define prev9_wd  (previous_words[8])
 
+	/* Macro for matching last keyword, case-insensitively. */
+#define LastKeywordMatches(keywords, pattern) \
+	(last_keyword_matches(previous_words, previous_words_count, keywords, pattern))
+
+	/* Macro for determining (loosely) which part of a DML query we are currently in. */
+#define CurrentQueryPartMatches(pattern) \
+	(LastKeywordMatches("SELECT|INSERT|UPDATE|DELETE|FROM|WHERE|GROUP|ORDER|HAVING", pattern))
+
 	/* Macros for matching the last N words before point, case-insensitively. */
 #define TailMatches1(p1) \
 	(previous_words_count >= 1 && \
@@ -1738,6 +1775,10 @@ psql_completion(const char *text, int start, int end)
 			matches = complete_from_variables(text, ":", "", true);
 	}
 
+	/* If current word ends with a comma, add a space. */
+	if (ends_with(text, ','))
+		COMPLETE_WITH_CONST(text);
+
 	/* If no previous word, suggest one of the basic sql commands */
 	else if (previous_words_count == 0)
 		COMPLETE_WITH_LIST(sql_commands);
@@ -3457,7 +3498,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH_CONST("IS");
 
 /* SELECT */
-	else if (TailMatches1("SELECT") || TailMatches2("SELECT", "ALL|DISTINCT"))
+	else if (HeadMatches1("SELECT|WITH") &&
+			 CurrentQueryPartMatches("SELECT") &&
+			 (ends_with(prev_wd, ',') || TailMatches1("SELECT|ALL|DISTINCT")))
 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
 							Query_addon_for_list_of_selectable_attributes);
 
#43Edmund Horner
ejrh00@gmail.com
In reply to: Joao De Almeida Pereira (#41)
Re: PATCH: psql tab completion for SELECT

On 17 July 2018 at 03:27, Joao De Almeida Pereira
<jdealmeidapereira@pivotal.io> wrote:

After playing alittle bit around with the patch I noticed that a comma was
missing in line 1214
+ 1202                 /* min_server_version */
+ 1203                 90000,
+ 1204                 /* catname */
+ 1205                 "pg_catalog.pg_proc p",
+ 1206                 /* selcondition */
+ 1207                 "p.prorettype NOT IN ('trigger'::regtype,
'internal'::regtype) "
+ 1208                 "AND 'internal'::regtype != ALL (p.proargtypes) "
+ 1209                 "AND p.oid NOT IN (SELECT
unnest(array[typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze])
FROM pg_type) "
+ 1210                 "AND p.oid NOT IN (SELECT
unnest(array[aggtransfn,aggfinalfn]) FROM pg_aggregate) "
+ 1211                 "AND p.oid NOT IN (SELECT
unnest(array[oprcode,oprrest,oprjoin]) FROM pg_operator) "
+ 1212                 "AND p.oid NOT IN (SELECT
unnest(array[lanplcallfoid,laninline,lanvalidator]) FROM pg_language) "
+ 1213                 "AND p.oid NOT IN (SELECT castfunc FROM pg_cast) "
+ 1214                 "AND p.oid NOT IN (SELECT amproc FROM pg_amproc) "
+ 1215                 /* viscondition */
+ 1216                 "pg_catalog.pg_function_is_visible(p.oid)",

To catch these typos would be good if we could get some testing around psql.
(Newbie question: do we have any kind of testing around tools like psql?)

Thanks
Joao

Hi Joao,

Ah yes, the embarrassing missing comma. I kind of wish my IDE or
compiler had pointed it out to me, but how was it to know that I
foolishly combining two struct fields? Sadly, I think the best
defence right now is to have others scrutinise the code.

I attached a new version of the patch in reply to Heikki. Would you
care to take a careful look for me?

I think some automated test framework for the tab completion queries
is possible, by calling psql_completion and examining the results.
Maybe someone will volunteer...

Edmund

#44Edmund Horner
ejrh00@gmail.com
In reply to: Edmund Horner (#1)
2 attachment(s)
Re: PATCH: psql tab completion for SELECT

Hi all,

Here are some rebased versions of the last two patches. No changes in
functionality, except a minor case sensitivity fix in the "completion
after commas" patch.

Edmund

Attachments:

01-psql-select-tab-completion-v8.patchapplication/octet-stream; name=01-psql-select-tab-completion-v8.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 2996006..d010f52 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -910,6 +910,63 @@ static const VersionedQuery Query_for_list_of_subscriptions[] = {
 	{0, NULL}
 };
 
+static const SchemaQuery Query_for_list_of_selectable_functions[] = {
+	/* For servers since 8.4, which introduced the unnest function. */
+	{
+		/* min_server_version */
+		80400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"NOT pg_catalog.arrayoverlap( "
+		"	ARRAY(SELECT oid FROM pg_catalog.pg_type WHERE typnamespace IN (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = 'pg_catalog') "
+		"		AND typname = ANY (ARRAY['internal', 'event_trigger', 'internal','opaque', 'unknown', 'pg_ddl_command', 'language_handler','tsm_handler', 'index_am_handler', 'fdw_handler'])), "
+		"	ARRAY(SELECT p.prorettype UNION SELECT pg_catalog.unnest(proargtypes)))",
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	/* For older servers since 7.4. */
+	{
+		/* min_server_version */
+		70400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* selcondition */
+		"prorettype NOT IN ('trigger'::regtype, 'internal'::regtype) ",
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{0, NULL}
+};
+
+/*
+ * This addon is used to find (unqualified) column names to include
+ * alongside the function names from the query above.
+ */
+static const VersionedQuery Query_addon_for_list_of_selectable_attributes[] = {
+	{70400,
+		"    UNION ALL "
+		"   SELECT DISTINCT pg_catalog.quote_ident(attname) "
+		"     FROM pg_catalog.pg_attribute "
+		"    WHERE attnum > 0 "
+		"      AND NOT attisdropped "
+		"      AND substring(pg_catalog.quote_ident(attname),1,%d)='%s'"
+	},
+	{0, NULL}
+};
+
 /*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
@@ -3110,7 +3167,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (TailMatches("SELECT") || TailMatches("SELECT", "ALL|DISTINCT"))
+		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
+											 Query_addon_for_list_of_selectable_attributes);
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
@@ -3720,7 +3779,8 @@ complete_from_versioned_schema_query(const char *text, int state)
  * where %d is the string length of the text and %s the text itself.
  *
  * If both simple_query and schema_query are non-NULL, then we construct
- * a schema query and append the (uninterpreted) string simple_query to it.
+ * a schema query and append the simple_query to it, replacing the %d and %s
+ * as described above.
  *
  * It is assumed that strings should be escaped to become SQL literals
  * (that is, what is in the query is actually ... '%s' ...)
@@ -3867,21 +3927,22 @@ _complete_from_query(const char *simple_query,
 							  " WHERE substring(pg_catalog.quote_ident(nspname) || '.',1,%d) ="
 							  " substring('%s',1,pg_catalog.length(pg_catalog.quote_ident(nspname))+1)) = 1",
 							  char_length, e_text);
-
-			/* If an addon query was provided, use it */
-			if (simple_query)
-				appendPQExpBuffer(&query_buffer, "\n%s", simple_query);
 		}
 		else
 		{
 			Assert(simple_query);
-			/* simple_query is an sprintf-style format string */
-			appendPQExpBuffer(&query_buffer, simple_query,
-							  char_length, e_text,
-							  e_info_charp, e_info_charp,
-							  e_info_charp2, e_info_charp2);
 		}
 
+		/*
+		 * simple_query is an sprintf-style format string (or it could be NULL, but
+		 * only if this is a schema query with no addon).
+		 */
+		if (simple_query)
+			appendPQExpBuffer(&query_buffer, simple_query,
+					  char_length, e_text,
+					  e_info_charp, e_info_charp,
+					  e_info_charp2, e_info_charp2);
+
 		/* Limit the number of records in the result */
 		appendPQExpBuffer(&query_buffer, "\nLIMIT %d",
 						  completion_max_records);
02-select-completion-after-commas.patchapplication/octet-stream; name=02-select-completion-after-commas.patchDownload
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index d010f52..62cc957 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -228,6 +228,13 @@ do { \
 	COMPLETE_WITH_LIST(list); \
 } while (0)
 
+#define COMPLETE_CURRENT_WORD() \
+do { \
+	completion_case_sensitive = true; \
+	completion_charp = text; \
+	matches = completion_matches(text, complete_from_const); \
+} while (0)
+
 #define COMPLETE_WITH_CS(...) \
 do { \
 	static const char *const list[] = { __VA_ARGS__, NULL }; \
@@ -1298,6 +1305,40 @@ ends_with(const char *s, char c)
 }
 
 /*
+ * Get the last keyword matching a pattern.
+ */
+static const char *
+last_keyword(int previous_words_count, char **previous_words,
+			 const char *keyword_pattern) {
+	int i;
+
+	for (i = 0; i < previous_words_count; i++) {
+		if (word_matches(keyword_pattern, previous_words[i], false)) {
+			return previous_words[i];
+		}
+	}
+
+	return NULL;
+}
+
+/*
+ * Was the last keyword one of the expected ones?  For example,
+ * LastKeywordMatchesImpl(...,
+ * 						  "SELECT|FROM|WHERE|GROUP|ORDER", "SELECT")
+ * is true if SELECT is the most recent of those keywords to appear.
+ */
+static bool
+LastKeywordMatchesImpl(int previous_words_count, char **previous_words,
+					   const char *keyword_pattern,
+					   const char *accepted_pattern) {
+	const char *last_kw = last_keyword(previous_words_count, previous_words,
+									   keyword_pattern);
+	if (!last_kw)
+		return false;
+	return word_matches(last_kw, accepted_pattern, false);
+}
+
+/*
  * The completion function.
  *
  * According to readline spec this gets passed the text entered so far and its
@@ -1367,6 +1408,18 @@ psql_completion(const char *text, int start, int end)
 	HeadMatchesImpl(true, previous_words_count, previous_words, \
 					VA_ARGS_NARGS(__VA_ARGS__), __VA_ARGS__)
 
+	/* Macro for matching last keyword, case-insensitively. */
+#define LastKeywordMatches(keywords, pattern) \
+	(LastKeywordMatchesImpl(previous_words_count, previous_words, \
+							keywords, pattern))
+
+	/*
+	 * Macro for determining (loosely) which part of a DML query we are
+	 * currently in.
+	 */
+#define CurrentQueryPartMatches(pattern) \
+	(LastKeywordMatches("SELECT|INSERT|UPDATE|DELETE|FROM|WHERE|GROUP|ORDER|HAVING", pattern))
+
 	/* Known command-starting keywords. */
 	static const char *const sql_commands[] = {
 		"ABORT", "ALTER", "ANALYZE", "BEGIN", "CALL", "CHECKPOINT", "CLOSE", "CLUSTER",
@@ -1449,6 +1502,13 @@ psql_completion(const char *text, int start, int end)
 			matches = complete_from_variables(text, ":", "", true);
 	}
 
+	/*
+	 * If current word ends with a comma, add a space; this will expedite
+	 * completions after commas for SELECT, etc.
+	 */
+	if (ends_with(text, ','))
+		COMPLETE_CURRENT_WORD();
+
 	/* If no previous word, suggest one of the basic sql commands */
 	else if (previous_words_count == 0)
 		COMPLETE_WITH_LIST(sql_commands);
@@ -3167,7 +3227,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH("IS");
 
 /* SELECT */
-	else if (TailMatches("SELECT") || TailMatches("SELECT", "ALL|DISTINCT"))
+	else if (HeadMatches("SELECT|WITH") &&
+			 CurrentQueryPartMatches("SELECT") &&
+			 (ends_with(prev_wd, ',') || TailMatches("SELECT|ALL|DISTINCT")))
 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
 											 Query_addon_for_list_of_selectable_attributes);
 
#45David Fetter
david@fetter.org
In reply to: Edmund Horner (#44)
2 attachment(s)
Re: PATCH: psql tab completion for SELECT

On Sat, Sep 22, 2018 at 06:53:55PM +1200, Edmund Horner wrote:

Hi all,

Here are some rebased versions of the last two patches. No changes in
functionality, except a minor case sensitivity fix in the "completion
after commas" patch.

Edmund

I've rebased and updated these to be more principled about what
functions could be tab completed.

Still missing: tests.

What precisely is this supposed to do?

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Attachments:

v9-0001-Infrastructure-for-tab-completion-in-the-SELECT-l.patchtext/x-diff; charset=us-asciiDownload
From 87fb63d70f5e08f853a2b8aa94e03c0d665f5000 Mon Sep 17 00:00:00 2001
From: David Fetter <david.fetter@aiven.io>
Date: Thu, 7 Oct 2021 15:59:06 -0700
Subject: [PATCH v9 1/2] Infrastructure for tab completion in the SELECT list

---
 src/bin/psql/tab-complete.c | 65 ++++++++++++++++++++++++++++++-------
 1 file changed, 54 insertions(+), 11 deletions(-)

diff --git src/bin/psql/tab-complete.c src/bin/psql/tab-complete.c
index ecae9df8ed..5db143523b 100644
--- src/bin/psql/tab-complete.c
+++ src/bin/psql/tab-complete.c
@@ -1017,6 +1017,45 @@ static const VersionedQuery Query_for_list_of_subscriptions[] = {
 	{0, NULL}
 };
 
+static const SchemaQuery Query_for_list_of_selectable_functions[] = {
+	{
+		/* min_server_version */
+		70400,
+		/* catname */
+		"pg_catalog.pg_proc p",
+		/* Disallow functions which take or return pseudotypes which are other than all* or *record */
+		"NOT((ARRAY[p.prorettype] || coalesce(p.proallargtypes, p.proargtypes)) "
+		"    && ARRAY(SELECT oid FROM pg_catalog.pg_type WHERE typtype='p' AND typname !~ '(^all|record$)'))"
+		/* Disallow stored procedures XXX this may change later. */
+		" AND p.prokind <> 'p'"
+		/* viscondition */
+		"pg_catalog.pg_function_is_visible(p.oid)",
+		/* namespace */
+		"p.pronamespace",
+		/* result */
+		"pg_catalog.quote_ident(p.proname)||'('",
+		/* qualresult */
+		NULL
+	},
+	{0, NULL}
+};
+
+/*
+ * This addon is used to find (unqualified) column names to include
+ * alongside the function names from the query above.
+ */
+static const VersionedQuery Query_addon_for_list_of_selectable_attributes[] = {
+	{70400,
+		"    UNION ALL "
+		"   SELECT DISTINCT pg_catalog.quote_ident(attname) "
+		"     FROM pg_catalog.pg_attribute "
+		"    WHERE attnum > 0 "
+		"      AND NOT attisdropped "
+		"      AND substring(pg_catalog.quote_ident(attname),1,%d)='%s'"
+	},
+	{0, NULL}
+};
+
 /*
  * This is a list of all "things" in Pgsql, which can show up after CREATE or
  * DROP; and there is also a query to get a list of them.
@@ -3768,7 +3807,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH("IS");
 
 /* SELECT */
-	/* naah . . . */
+	else if (TailMatches("SELECT") || TailMatches("SELECT", "ALL|DISTINCT"))
+		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
+											 Query_addon_for_list_of_selectable_attributes);
 
 /* SET, RESET, SHOW */
 	/* Complete with a variable name */
@@ -4432,7 +4473,8 @@ complete_from_versioned_schema_query(const char *text, int state)
  * where %d is the string length of the text and %s the text itself.
  *
  * If both simple_query and schema_query are non-NULL, then we construct
- * a schema query and append the (uninterpreted) string simple_query to it.
+ * a schema query and append the simple_query to it, replacing the %d and %s
+ * as described above.
  *
  * It is assumed that strings should be escaped to become SQL literals
  * (that is, what is in the query is actually ... '%s' ...)
@@ -4579,21 +4621,22 @@ _complete_from_query(const char *simple_query,
 							  " WHERE substring(pg_catalog.quote_ident(nspname) || '.',1,%d) ="
 							  " substring('%s',1,pg_catalog.length(pg_catalog.quote_ident(nspname))+1)) = 1",
 							  char_length, e_text);
-
-			/* If an addon query was provided, use it */
-			if (simple_query)
-				appendPQExpBuffer(&query_buffer, "\n%s", simple_query);
 		}
 		else
 		{
 			Assert(simple_query);
-			/* simple_query is an sprintf-style format string */
-			appendPQExpBuffer(&query_buffer, simple_query,
-							  char_length, e_text,
-							  e_info_charp, e_info_charp,
-							  e_info_charp2, e_info_charp2);
 		}
 
+		/*
+		 * simple_query is an sprintf-style format string (or it could be NULL, but
+		 * only if this is a schema query with no addon).
+		 */
+		if (simple_query)
+			appendPQExpBuffer(&query_buffer, simple_query,
+					  char_length, e_text,
+					  e_info_charp, e_info_charp,
+					  e_info_charp2, e_info_charp2);
+
 		/* Limit the number of records in the result */
 		appendPQExpBuffer(&query_buffer, "\nLIMIT %d",
 						  completion_max_records);
-- 
2.31.1

v9-0002-SELECT-completion-after-commas.patchtext/x-diff; charset=us-asciiDownload
From 3764b9fe211db1fa322fa83103d39356928942ae Mon Sep 17 00:00:00 2001
From: David Fetter <david.fetter@aiven.io>
Date: Thu, 7 Oct 2021 18:56:52 -0700
Subject: [PATCH v9 2/2] SELECT completion after commas

---
 src/bin/psql/tab-complete.c | 64 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 63 insertions(+), 1 deletion(-)

diff --git src/bin/psql/tab-complete.c src/bin/psql/tab-complete.c
index 5db143523b..6fa4af08e0 100644
--- src/bin/psql/tab-complete.c
+++ src/bin/psql/tab-complete.c
@@ -247,6 +247,13 @@ do { \
 	COMPLETE_WITH_LIST(list); \
 } while (0)
 
+#define COMPLETE_CURRENT_WORD() \
+do { \
+	completion_case_sensitive = true; \
+	completion_charp = text; \
+	matches = rl_completion_matches(text, complete_from_const); \
+} while (0)
+
 #define COMPLETE_WITH_CS(...) \
 do { \
 	static const char *const list[] = { __VA_ARGS__, NULL }; \
@@ -1459,6 +1466,40 @@ ends_with(const char *s, char c)
 	return (length > 0 && s[length - 1] == c);
 }
 
+/*
+ * Get the last keyword matching a pattern.
+ */
+static const char *
+last_keyword(int previous_words_count, char **previous_words,
+			 const char *keyword_pattern)
+{
+	for (int i = 0; i < previous_words_count; i++)
+	{
+		if (word_matches(keyword_pattern, previous_words[i], false))
+			return previous_words[i];
+	}
+
+	return NULL;
+}
+
+/*
+ * Was the last keyword one of the expected ones?  For example,
+ * LastKeywordMatchesImpl(...,
+ * 						  "SELECT|FROM|WHERE|GROUP|ORDER", "SELECT")
+ * is true if SELECT is the most recent of those keywords to appear.
+ */
+static bool
+LastKeywordMatchesImpl(int previous_words_count, char **previous_words,
+					   const char *keyword_pattern,
+					   const char *accepted_pattern)
+{
+	const char *last_kw = last_keyword(previous_words_count, previous_words,
+									   keyword_pattern);
+	if (!last_kw)
+		return false;
+	return word_matches(last_kw, accepted_pattern, false);
+}
+
 /*
  * The completion function.
  *
@@ -1529,6 +1570,18 @@ psql_completion(const char *text, int start, int end)
 	HeadMatchesImpl(true, previous_words_count, previous_words, \
 					VA_ARGS_NARGS(__VA_ARGS__), __VA_ARGS__)
 
+	/* Macro for matching last keyword, case-insensitively. */
+#define LastKeywordMatches(keywords, pattern) \
+	(LastKeywordMatchesImpl(previous_words_count, previous_words, \
+							keywords, pattern))
+
+	/*
+	 * Macro for determining (loosely) which part of a DML query we are
+	 * currently in.
+	 */
+#define CurrentQueryPartMatches(pattern) \
+	(LastKeywordMatches("SELECT|INSERT|UPDATE|DELETE|FROM|WHERE|GROUP|ORDER|HAVING", pattern))
+
 	/* Known command-starting keywords. */
 	static const char *const sql_commands[] = {
 		"ABORT", "ALTER", "ANALYZE", "BEGIN", "CALL", "CHECKPOINT", "CLOSE", "CLUSTER",
@@ -1622,6 +1675,13 @@ psql_completion(const char *text, int start, int end)
 			matches = complete_from_variables(text, ":", "", true);
 	}
 
+	/*
+	 * If current word ends with a comma, add a space; this will expedite
+	 * completions after commas for SELECT, etc.
+	 */
+	if (ends_with(text, ','))
+		COMPLETE_CURRENT_WORD();
+
 	/* If no previous word, suggest one of the basic sql commands */
 	else if (previous_words_count == 0)
 		COMPLETE_WITH_LIST(sql_commands);
@@ -3807,7 +3867,9 @@ psql_completion(const char *text, int start, int end)
 		COMPLETE_WITH("IS");
 
 /* SELECT */
-	else if (TailMatches("SELECT") || TailMatches("SELECT", "ALL|DISTINCT"))
+	else if (HeadMatches("SELECT|WITH") &&
+			 CurrentQueryPartMatches("SELECT") &&
+			 (ends_with(prev_wd, ',') || TailMatches("SELECT|ALL|DISTINCT")))
 		COMPLETE_WITH_VERSIONED_SCHEMA_QUERY(Query_for_list_of_selectable_functions,
 											 Query_addon_for_list_of_selectable_attributes);
 
-- 
2.31.1

#46Edmund Horner
ejrh00@gmail.com
In reply to: David Fetter (#45)
Re: PATCH: psql tab completion for SELECT

On Fri, 8 Oct 2021 at 20:01, David Fetter <david@fetter.org> wrote:

On Sat, Sep 22, 2018 at 06:53:55PM +1200, Edmund Horner wrote:

Hi all,

Here are some rebased versions of the last two patches. No changes in
functionality, except a minor case sensitivity fix in the "completion
after commas" patch.

Edmund

I've rebased and updated these to be more principled about what
functions could be tab completed.

Still missing: tests.

What precisely is this supposed to do?

Hi David,

The main patch 0001 was to add completion for "SELECT <tab>" using
attributes, functions, and a couple of hard-coded options.

The changes to _complete_from_query were so that simple_query (when used in
addon mode, as for this feature) could have the current word interpolated
into it. This feature uses a schema query to get the function names, as
they can be schema qualified, and an addon to get the column names, as they
can't (they could be table-qualified, but that's hard when you don't know
what the table aliases will be). Previously existing queries using addons
did not need to interpolate into them, but this one did, hence the change.

The second patch 0002 was to build on 0001 and add support for pressing
<tab> after a comma while in the column list, i.e. when SELECT was the most
recent major keyword (major being defined by the list of keywords in
CurrentQueryPartMatches).

There's a bit of trickery around completing "," by adding a single space.
Without the space, the next <tab> will try to complete using the whole word
including the part before the comma.

Regarding testing, I can't see any automated tests for tab completion.
Though it seems to me we could do it with a readline driver program of some
sorts, if the current regression test scripts don't work interactively.

Cheers,
Edmund

#47Tom Lane
tgl@sss.pgh.pa.us
In reply to: Edmund Horner (#46)
Re: PATCH: psql tab completion for SELECT

Edmund Horner <ejrh00@gmail.com> writes:

On Fri, 8 Oct 2021 at 20:01, David Fetter <david@fetter.org> wrote:

What precisely is this supposed to do?

The main patch 0001 was to add completion for "SELECT <tab>" using
attributes, functions, and a couple of hard-coded options.

This sort of thing has been proposed a few times, but never made it to
commit, because there's basically no way to limit the completions to a
usefully small set of possibilities. SQL wasn't designed to make this
easy :-(

Regarding testing, I can't see any automated tests for tab completion.

src/bin/psql/t/010_tab_completion.pl

regards, tom lane