new set of psql patches for loading (saving) data from (to) text, binary files
Hi
I am sending set of patches - for simpler testing these patches are
independent in this moment.
These patches are replacement of my previous patches in this area: COPY RAW
and fileref variables.
1. parametrized queries support - the psql variables can be passed as query
parameters
2. \gstore, \gbstore - save returned (binary) value to file
3. \set_from_file. \set_from_bfile - set a variable from (binary) file
The code is simple - there are not any change in critical or complex parts
of psql.
Regards
Pavel
Comments, notes?
Attachments:
psql-gstore-01.patchtext/x-patch; charset=US-ASCII; name=psql-gstore-01.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 9915731..7e2fa96 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -1944,6 +1944,31 @@ hello 10
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>\gstore [ <replaceable class="parameter">filename</replaceable> ]</literal></term>
+ <term><literal>\gstore [ |<replaceable class="parameter">command</replaceable> ]</literal></term>
+ <term><literal>\gbstore [ <replaceable class="parameter">filename</replaceable> ]</literal></term>
+ <term><literal>\gbstore [ |<replaceable class="parameter">command</replaceable> ]</literal></term>
+ <listitem>
+ <para>
+ Sends the current query input buffer to the server and stores the
+ raw query's output into stores the query's output in <replaceable
+ class="parameter">filename</replaceable> or pipes the output
+ to the shell command <replaceable
+ class="parameter">command</replaceable>. The file or command is
+ written to only if the query successfully returns exactly one row
+ one column non null result, not if the query fails or is a
+ non-data-returning SQL command. For example:
+<programlisting>
+=> <userinput>SELECT avatar FROM users WHERE id = 123</userinput>
+-> <userinput>\gbstore ~/avatar.png</userinput>
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
+
+
<varlistentry>
<term><literal>\h</literal> or <literal>\help</literal> <literal>[ <replaceable class="parameter">command</replaceable> ]</literal></term>
<listitem>
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a9a2fdb..e8fabb9 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -929,6 +929,27 @@ exec_command(const char *cmd,
status = PSQL_CMD_SEND;
}
+ /* \gstore [filename], \gbstore [filename] -- send query and store result in (binary) file */
+ else if (strcmp(cmd, "gstore") == 0 ||
+ (strcmp(cmd, "gbstore") == 0))
+ {
+ char *fname = psql_scan_slash_option(scan_state,
+ OT_FILEPIPE, NULL, false);
+
+ if (!fname)
+ pset.gfname = pg_strdup("");
+ else
+ {
+ expand_tilde(&fname);
+ pset.gfname = pg_strdup(fname);
+ }
+
+ pset.raw_flag = true;
+ pset.binres_flag = (strcmp(cmd, "gbstore") == 0);
+ free(fname);
+ status = PSQL_CMD_SEND;
+ }
+
/* help */
else if (strcmp(cmd, "h") == 0 || strcmp(cmd, "help") == 0)
{
@@ -1064,7 +1085,6 @@ exec_command(const char *cmd,
free(opt2);
}
-
/* \o -- set query output */
else if (strcmp(cmd, "o") == 0 || strcmp(cmd, "out") == 0)
{
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index a7789df..d4b4f15 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -854,6 +854,85 @@ StoreQueryTuple(const PGresult *result)
return success;
}
+/*
+ * StoreRawResult: the returned value (possibly binary) is displayed
+ * or stored in file. The result should be exactly one row, one column.
+ */
+static bool
+StoreRawResult(const PGresult *result)
+{
+ bool success = true;
+
+ if (PQntuples(result) < 1)
+ {
+ psql_error("no rows returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQntuples(result) > 1)
+ {
+ psql_error("more than one row returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQnfields(result) < 1)
+ {
+ psql_error("no columns returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQnfields(result) > 1)
+ {
+ psql_error("more than one column returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQgetisnull(result, 0, 0))
+ {
+ psql_error("returned value is null for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else
+ {
+ char *value;
+ int length;
+ FILE *fout = NULL;
+ bool is_pipe = false;
+
+ value = PQgetvalue(result, 0, 0);
+ length = PQgetlength(result, 0, 0);
+
+ if (pset.gfname && *(pset.gfname) != '\0')
+ {
+ if (!openQueryOutputFile(pset.gfname, &fout, &is_pipe))
+ success = false;
+ if (success && is_pipe)
+ disable_sigpipe_trap();
+ }
+
+ if (success)
+ {
+ success = fwrite(value, 1, length, fout != NULL ? fout : pset.queryFout) == length;
+ if (!success)
+ psql_error("%s: %s\n", pset.gfname, strerror(errno));
+
+ if (success)
+ success = fflush(fout != NULL ? fout : pset.queryFout) == 0;
+
+ if (!success)
+ psql_error("%s: %s\n", pset.gfname, strerror(errno));
+
+ if (fout != NULL)
+ {
+ if (is_pipe)
+ {
+ pclose(fout);
+ restore_sigpipe_trap();
+ }
+ else
+ fclose(fout);
+ }
+ }
+ }
+
+ return success;
+}
/*
* ExecQueryTuples: assuming query result is OK, execute each query
@@ -1124,6 +1203,8 @@ PrintQueryResults(PGresult *results)
success = ExecQueryTuples(results);
else if (pset.crosstab_flag)
success = PrintResultsInCrosstab(results);
+ else if (pset.raw_flag)
+ success = StoreRawResult(results);
else
success = PrintQueryTuples(results);
/* if it's INSERT/UPDATE/DELETE RETURNING, also print status */
@@ -1278,7 +1359,8 @@ SendQuery(const char *query)
}
if (pset.fetch_count <= 0 || pset.gexec_flag ||
- pset.crosstab_flag || !is_select_command(query))
+ pset.crosstab_flag || !is_select_command(query) ||
+ pset.raw_flag)
{
/* Default fetch-it-all-and-print mode */
instr_time before,
@@ -1287,7 +1369,16 @@ SendQuery(const char *query)
if (pset.timing)
INSTR_TIME_SET_CURRENT(before);
- results = PQexec(pset.db, query);
+ if (pset.binres_flag)
+ results = PQexecParams(pset.db, query,
+ 0,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ pset.binres_flag);
+ else
+ results = PQexec(pset.db, query);
/* these operations are included in the timing result: */
ResetCancelConn();
@@ -1404,7 +1495,7 @@ SendQuery(const char *query)
sendquery_cleanup:
- /* reset \g's output-to-filename trigger */
+ /* reset \g, \g[b]store output-to-filename trigger */
if (pset.gfname)
{
free(pset.gfname);
@@ -1421,6 +1512,10 @@ sendquery_cleanup:
/* reset \gexec trigger */
pset.gexec_flag = false;
+ /* reset \gstore, gbstore trigger */
+ pset.raw_flag = false;
+ pset.binres_flag = false;
+
/* reset \crosstabview trigger */
pset.crosstab_flag = false;
for (i = 0; i < lengthof(pset.ctv_args); i++)
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a69c4dd..7f337f9 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -168,7 +168,7 @@ slashUsage(unsigned short int pager)
* Use "psql --help=commands | wc" to count correctly. It's okay to count
* the USE_READLINE line even in builds without that.
*/
- output = PageOutput(113, pager ? &(pset.popt.topt) : NULL);
+ output = PageOutput(115, pager ? &(pset.popt.topt) : NULL);
fprintf(output, _("General\n"));
fprintf(output, _(" \\copyright show PostgreSQL usage and distribution terms\n"));
@@ -176,6 +176,8 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\g [FILE] or ; execute query (and send results to file or |pipe)\n"));
fprintf(output, _(" \\gexec execute query, then execute each value in its result\n"));
fprintf(output, _(" \\gset [PREFIX] execute query and store results in psql variables\n"));
+ fprintf(output, _(" \\gstore [FILE] execute query and store result to file or |pipe\n"));
+ fprintf(output, _(" \\gbstore [FILE] execute query and store bin result to file or |pipe\n"));
fprintf(output, _(" \\q quit psql\n"));
fprintf(output, _(" \\crosstabview [COLUMNS] execute query and display results in crosstab\n"));
fprintf(output, _(" \\watch [SEC] execute query every SEC seconds\n"));
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 8cfe9d2..74a99e6 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -95,6 +95,8 @@ typedef struct _psqlSettings
bool gexec_flag; /* one-shot flag to execute query's results */
bool crosstab_flag; /* one-shot request to crosstab results */
char *ctv_args[4]; /* \crosstabview arguments */
+ bool raw_flag; /* one-shot flag to work with exact one value */
+ bool binres_flag; /* one-shot flag - enforce binary result format */
bool notty; /* stdin or stdout is not a tty (as determined
* on startup) */
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index cd64c39..166e3a7 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1336,7 +1336,8 @@ psql_completion(const char *text, int start, int end)
"\\dm", "\\dn", "\\do", "\\dO", "\\dp", "\\drds", "\\ds", "\\dS",
"\\dt", "\\dT", "\\dv", "\\du", "\\dx", "\\dy",
"\\e", "\\echo", "\\ef", "\\encoding", "\\errverbose", "\\ev",
- "\\f", "\\g", "\\gexec", "\\gset", "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
+ "\\f", "\\g", "\\gbstore", "\\gexec", "\\gset", "gstore",
+ "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
"\\lo_import", "\\lo_export", "\\lo_list", "\\lo_unlink",
"\\o", "\\p", "\\password", "\\prompt", "\\pset", "\\q", "\\qecho", "\\r",
"\\s", "\\set", "\\setenv", "\\sf", "\\sv", "\\t", "\\T",
@@ -3240,8 +3241,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
else if (TailMatchesCS1("\\sv*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
- else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\i|\\include|"
- "\\ir|\\include_relative|\\o|\\out|"
+ else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\gbstore|\\gstore|"
+ "\\i|\\include|\\ir|\\include_relative|\\o|\\out|"
"\\s|\\w|\\write|\\lo_import"))
{
completion_charp = "\\";
psql-paramatrized_queries-01.patchtext/x-patch; charset=US-ASCII; name=psql-paramatrized_queries-01.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 9915731..b16670d 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3353,6 +3353,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>PARAMETRIZED_QUERIES</varname></term>
+ <listitem>
+ <para>
+ The psql's variables can be injected to query text (by default) or
+ passed as query parameters when this variable is set
+ <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>PORT</varname></term>
<listitem>
<para>
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index a7789df..b20f8f5 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -119,9 +119,13 @@ setQFout(const char *fname)
* If "escape" is true, return the value suitably quoted and escaped,
* as an identifier or string literal depending on "as_ident".
* (Failure in escaping should lead to returning NULL.)
+ *
+ * When "inside_query" is true, then the variable can be passed as query parameter,
+ * when it is not used as identifier (as_ident:false), when escape is not required
+ * (escaping changes the content).
*/
char *
-psql_get_variable(const char *varname, bool escape, bool as_ident)
+psql_get_variable(const char *varname, bool escape, bool as_ident, bool inside_query)
{
char *result;
const char *value;
@@ -130,6 +134,35 @@ psql_get_variable(const char *varname, bool escape, bool as_ident)
if (!value)
return NULL;
+ if (inside_query && pset.parametrized_queries)
+ {
+ if (!escape && !as_ident)
+ {
+ char printbuf[10];
+
+ if (pset.nparams >= pset.max_params)
+ {
+ /* create or realloc params array */
+ if (pset.max_params > 0)
+ {
+ pset.max_params += 16;
+ pset.params = (const char **) pg_realloc(pset.params,
+ sizeof(const char *) * pset.max_params);
+ }
+ else
+ {
+ pset.max_params = 16;
+ pset.params = (const char **) pg_malloc(sizeof(const char *) * pset.max_params);
+ }
+ }
+
+ pset.params[pset.nparams++] = value;
+ snprintf(printbuf, sizeof(printbuf) - 1, "$%d", pset.nparams);
+
+ return pstrdup(printbuf);
+ }
+ }
+
if (escape)
{
char *escaped_value;
@@ -1287,7 +1320,16 @@ SendQuery(const char *query)
if (pset.timing)
INSTR_TIME_SET_CURRENT(before);
- results = PQexec(pset.db, query);
+ if (pset.nparams > 0)
+ results = PQexecParams(pset.db, query,
+ pset.nparams,
+ NULL,
+ (const char * const *) pset.params,
+ NULL,
+ NULL,
+ 0);
+ else
+ results = PQexec(pset.db, query);
/* these operations are included in the timing result: */
ResetCancelConn();
@@ -1382,6 +1424,15 @@ SendQuery(const char *query)
ClearOrSaveResult(results);
+ /* the number of query parameters are not necessary now */
+ pset.nparams = 0;
+ if (pset.max_params > 0)
+ {
+ free(pset.params);
+ pset.params = NULL;
+ pset.max_params = 0;
+ }
+
/* Possible microtiming output */
if (pset.timing)
PrintTiming(elapsed_msec);
@@ -1488,7 +1539,16 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
appendPQExpBuffer(&buf, "DECLARE _psql_cursor NO SCROLL CURSOR FOR\n%s",
query);
- results = PQexec(pset.db, buf.data);
+ if (pset.nparams > 0)
+ results = PQexecParams(pset.db, buf.data,
+ pset.nparams,
+ NULL,
+ (const char * const *) pset.params,
+ NULL,
+ NULL,
+ 0);
+ else
+ results = PQexec(pset.db, buf.data);
OK = AcceptResult(results) &&
(PQresultStatus(results) == PGRES_COMMAND_OK);
ClearOrSaveResult(results);
diff --git a/src/bin/psql/common.h b/src/bin/psql/common.h
index bdcb58f..6d8eda7 100644
--- a/src/bin/psql/common.h
+++ b/src/bin/psql/common.h
@@ -18,7 +18,7 @@
extern bool openQueryOutputFile(const char *fname, FILE **fout, bool *is_pipe);
extern bool setQFout(const char *fname);
-extern char *psql_get_variable(const char *varname, bool escape, bool as_ident);
+extern char *psql_get_variable(const char *varname, bool escape, bool as_ident, bool inside_query);
extern void psql_error(const char *fmt,...) pg_attribute_printf(1, 2);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a69c4dd..3c62146 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -325,7 +325,7 @@ helpVariables(unsigned short int pager)
* Windows builds currently print one more line than non-Windows builds.
* Using the larger number is fine.
*/
- output = PageOutput(88, pager ? &(pset.popt.topt) : NULL);
+ output = PageOutput(90, pager ? &(pset.popt.topt) : NULL);
fprintf(output, _("List of specially treated variables\n\n"));
@@ -352,6 +352,8 @@ helpVariables(unsigned short int pager)
fprintf(output, _(" LASTOID value of the last affected OID\n"));
fprintf(output, _(" ON_ERROR_ROLLBACK if set, an error doesn't stop a transaction (uses implicit savepoints)\n"));
fprintf(output, _(" ON_ERROR_STOP stop batch execution after error\n"));
+ fprintf(output, _(" PARAMETRIZED_QUERIES\n"
+ " pass psql's variables as query parameters\n"));
fprintf(output, _(" PORT server port of the current connection\n"));
fprintf(output, _(" PROMPT1 specifies the standard psql prompt\n"));
fprintf(output, _(" PROMPT2 specifies the prompt used when a statement continues from a previous line\n"));
diff --git a/src/bin/psql/mainloop.c b/src/bin/psql/mainloop.c
index 37dfa4d..9638a5b 100644
--- a/src/bin/psql/mainloop.c
+++ b/src/bin/psql/mainloop.c
@@ -403,6 +403,15 @@ MainLoop(FILE *source)
psql_scan_finish(scan_state);
free(line);
+ /* reset a number of query parameters */
+ pset.nparams = 0;
+ if (pset.max_params > 0)
+ {
+ free(pset.params);
+ pset.params = NULL;
+ pset.max_params = 0;
+ }
+
if (slashCmdStatus == PSQL_CMD_TERMINATE)
{
successResult = EXIT_SUCCESS;
diff --git a/src/bin/psql/psqlscanslash.l b/src/bin/psql/psqlscanslash.l
index 86832a8..4a34f29 100644
--- a/src/bin/psql/psqlscanslash.l
+++ b/src/bin/psql/psqlscanslash.l
@@ -243,6 +243,7 @@ other .
yyleng - 1);
value = cur_state->callbacks->get_variable(varname,
false,
+ false,
false);
free(varname);
@@ -271,7 +272,7 @@ other .
ECHO;
else
{
- psqlscan_escape_variable(cur_state, yytext, yyleng, false);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, false, false);
*option_quote = ':';
}
unquoted_option_chars = 0;
@@ -283,7 +284,7 @@ other .
ECHO;
else
{
- psqlscan_escape_variable(cur_state, yytext, yyleng, true);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, true, false);
*option_quote = ':';
}
unquoted_option_chars = 0;
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 8cfe9d2..4e75bd8 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -120,6 +120,7 @@ typedef struct _psqlSettings
* functions.
*/
bool autocommit;
+ bool parametrized_queries;
bool on_error_stop;
bool quiet;
bool singleline;
@@ -135,6 +136,9 @@ typedef struct _psqlSettings
const char *prompt3;
PGVerbosity verbosity; /* current error verbosity level */
PGContextVisibility show_context; /* current context display level */
+ int nparams; /* number of query parameters */
+ int max_params; /* max size of current parameters array */
+ const char **params; /* query parameters */
} PsqlSettings;
extern PsqlSettings pset;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 7ce05fb..6059e44 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -139,6 +139,10 @@ main(int argc, char *argv[])
pset.last_error_result = NULL;
pset.cur_cmd_source = stdin;
pset.cur_cmd_interactive = false;
+ pset.parametrized_queries = false;
+ pset.nparams = 0;
+ pset.max_params = 0;
+ pset.params = NULL;
/* We rely on unmentioned fields of pset.popt to start out 0/false/NULL */
pset.popt.topt.format = PRINT_ALIGNED;
@@ -793,6 +797,12 @@ autocommit_hook(const char *newval)
}
static void
+parametrized_queries_hook(const char *newval)
+{
+ pset.parametrized_queries = ParseVariableBool(newval, "PARAMETRIZED_QUERIES");
+}
+
+static void
on_error_stop_hook(const char *newval)
{
pset.on_error_stop = ParseVariableBool(newval, "ON_ERROR_STOP");
@@ -990,6 +1000,7 @@ EstablishVariableSpace(void)
SetVariableAssignHook(pset.vars, "ON_ERROR_ROLLBACK", on_error_rollback_hook);
SetVariableAssignHook(pset.vars, "COMP_KEYWORD_CASE", comp_keyword_case_hook);
SetVariableAssignHook(pset.vars, "HISTCONTROL", histcontrol_hook);
+ SetVariableAssignHook(pset.vars, "PARAMETRIZED_QUERIES", parametrized_queries_hook);
SetVariableAssignHook(pset.vars, "PROMPT1", prompt1_hook);
SetVariableAssignHook(pset.vars, "PROMPT2", prompt2_hook);
SetVariableAssignHook(pset.vars, "PROMPT3", prompt3_hook);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index cd64c39..8a38671 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3216,8 +3216,8 @@ psql_completion(const char *text, int start, int end)
}
else if (TailMatchesCS2("\\set", MatchAny))
{
- if (TailMatchesCS1("AUTOCOMMIT|ON_ERROR_STOP|QUIET|"
- "SINGLELINE|SINGLESTEP"))
+ if (TailMatchesCS1("AUTOCOMMIT|ON_ERROR_STOP|PARAMETRIZED_QUERIES|"
+ "QUIET|SINGLELINE|SINGLESTEP"))
COMPLETE_WITH_LIST_CS2("on", "off");
else if (TailMatchesCS1("COMP_KEYWORD_CASE"))
COMPLETE_WITH_LIST_CS4("lower", "upper",
@@ -3709,7 +3709,7 @@ complete_from_variables(const char *text, const char *prefix, const char *suffix
"AUTOCOMMIT", "COMP_KEYWORD_CASE", "DBNAME", "ECHO", "ECHO_HIDDEN",
"ENCODING", "FETCH_COUNT", "HISTCONTROL", "HISTFILE", "HISTSIZE",
"HOST", "IGNOREEOF", "LASTOID", "ON_ERROR_ROLLBACK", "ON_ERROR_STOP",
- "PORT", "PROMPT1", "PROMPT2", "PROMPT3", "QUIET",
+ "PARAMETRIZED_QUERIES", "PORT", "PROMPT1", "PROMPT2", "PROMPT3", "QUIET",
"SHOW_CONTEXT", "SINGLELINE", "SINGLESTEP",
"USER", "VERBOSITY", NULL
};
diff --git a/src/fe_utils/psqlscan.l b/src/fe_utils/psqlscan.l
index 55067b4..06a6519 100644
--- a/src/fe_utils/psqlscan.l
+++ b/src/fe_utils/psqlscan.l
@@ -700,7 +700,8 @@ other .
if (cur_state->callbacks->get_variable)
value = cur_state->callbacks->get_variable(varname,
false,
- false);
+ false,
+ true);
else
value = NULL;
@@ -736,11 +737,11 @@ other .
}
:'{variable_char}+' {
- psqlscan_escape_variable(cur_state, yytext, yyleng, false);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, false, true);
}
:\"{variable_char}+\" {
- psqlscan_escape_variable(cur_state, yytext, yyleng, true);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, true, true);
}
/*
@@ -1401,7 +1402,7 @@ psqlscan_extract_substring(PsqlScanState state, const char *txt, int len)
*/
void
psqlscan_escape_variable(PsqlScanState state, const char *txt, int len,
- bool as_ident)
+ bool as_ident, bool from_query)
{
char *varname;
char *value;
@@ -1409,7 +1410,8 @@ psqlscan_escape_variable(PsqlScanState state, const char *txt, int len,
/* Variable lookup. */
varname = psqlscan_extract_substring(state, txt + 2, len - 3);
if (state->callbacks->get_variable)
- value = state->callbacks->get_variable(varname, true, as_ident);
+ value = state->callbacks->get_variable(varname,
+ true, as_ident, from_query);
else
value = NULL;
free(varname);
diff --git a/src/include/fe_utils/psqlscan.h b/src/include/fe_utils/psqlscan.h
index 1f10ecc..3854117 100644
--- a/src/include/fe_utils/psqlscan.h
+++ b/src/include/fe_utils/psqlscan.h
@@ -53,7 +53,8 @@ typedef struct PsqlScanCallbacks
{
/* Fetch value of a variable, as a pfree'able string; NULL if unknown */
/* This pointer can be NULL if no variable substitution is wanted */
- char *(*get_variable) (const char *varname, bool escape, bool as_ident);
+ char *(*get_variable) (const char *varname,
+ bool escape, bool as_ident, bool from_query);
/* Print an error message someplace appropriate */
/* (very old gcc versions don't support attributes on function pointers) */
#if defined(__GNUC__) && __GNUC__ < 4
diff --git a/src/include/fe_utils/psqlscan_int.h b/src/include/fe_utils/psqlscan_int.h
index a52929d..53210b2 100644
--- a/src/include/fe_utils/psqlscan_int.h
+++ b/src/include/fe_utils/psqlscan_int.h
@@ -139,6 +139,7 @@ extern char *psqlscan_extract_substring(PsqlScanState state,
const char *txt, int len);
extern void psqlscan_escape_variable(PsqlScanState state,
const char *txt, int len,
- bool as_ident);
+ bool as_ident,
+ bool from_query);
#endif /* PSQLSCAN_INT_H */
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 464436a..7fed568 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2714,3 +2714,36 @@ NOTICE: foo
CONTEXT: PL/pgSQL function inline_code_block line 3 at RAISE
ERROR: bar
CONTEXT: PL/pgSQL function inline_code_block line 4 at RAISE
+-- parametrized queries
+\set PARAMETRIZED_QUERIES off
+\set a1 'AHOJ SVETE'
+-- should fail
+SELECT :a1;
+ERROR: column "ahoj" does not exist
+LINE 1: SELECT AHOJ SVETE;
+ ^
+-- ok
+SELECT :'a1';
+ ?column?
+------------
+ AHOJ SVETE
+(1 row)
+
+\set PARAMETRIZED_QUERIES on
+-- should fail - unknown type
+SELECT :a1;
+ERROR: could not determine data type of parameter $1
+-- ok
+SELECT :a1::text;
+ text
+------------
+ AHOJ SVETE
+(1 row)
+
+-- returns true, when value passed as parameter is same as client side evaluated variable
+SELECT :a1 = :'a1';
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/sql/psql.sql b/src/test/regress/sql/psql.sql
index 900aa7e..69e2df1 100644
--- a/src/test/regress/sql/psql.sql
+++ b/src/test/regress/sql/psql.sql
@@ -379,3 +379,23 @@ begin
raise notice 'foo';
raise exception 'bar';
end $$;
+
+-- parametrized queries
+\set PARAMETRIZED_QUERIES off
+\set a1 'AHOJ SVETE'
+
+-- should fail
+SELECT :a1;
+
+-- ok
+SELECT :'a1';
+
+\set PARAMETRIZED_QUERIES on
+-- should fail - unknown type
+SELECT :a1;
+
+-- ok
+SELECT :a1::text;
+
+-- returns true, when value passed as parameter is same as client side evaluated variable
+SELECT :a1 = :'a1';
psql-set-from-file-01.patchtext/x-patch; charset=US-ASCII; name=psql-set-from-file-01.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 9915731..1d77569 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -2697,6 +2697,33 @@ lo_import 152801
<varlistentry>
+ <term><literal>\set_from_bfile <replaceable class="parameter">name</replaceable> <replaceable class="parameter">filename</replaceable></literal></term>
+
+ <listitem>
+ <para>
+ Sets the <application>psql</> variable <replaceable
+ class="parameter">name</replaceable> by content of
+ binary file <replaceable class="parameter">filename</replaceable>.
+ The content is escapeaed as bytea value.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term><literal>\set_from_file <replaceable class="parameter">name</replaceable> <replaceable class="parameter">filename</replaceable></literal></term>
+
+ <listitem>
+ <para>
+ Sets the <application>psql</> variable <replaceable
+ class="parameter">name</replaceable> by content of
+ text file <replaceable class="parameter">filename</replaceable>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
<term><literal>\setenv <replaceable class="parameter">name</replaceable> [ <replaceable class="parameter">value</replaceable> ]</literal></term>
<listitem>
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a9a2fdb..bcc1793 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -82,6 +82,7 @@ static void minimal_error_message(PGresult *res);
static void printSSLInfo(void);
static bool printPsetInfo(const char *param, struct printQueryOpt *popt);
static char *pset_value_string(const char *param, struct printQueryOpt *popt);
+static bool read_file(char *fname, PQExpBuffer rawbuf);
#ifdef WIN32
static void checkWin32Codepage(void);
@@ -1318,6 +1319,82 @@ exec_command(const char *cmd,
free(opt0);
}
+ /* \set_from_file, \set_from_bfile -- set variable/option command from file */
+ else if (strcmp(cmd, "set_from_file") == 0 || strcmp(cmd, "set_from_bfile") == 0)
+ {
+ char *varname = psql_scan_slash_option(scan_state,
+ OT_NORMAL, NULL, false);
+
+ if (!varname)
+ {
+ psql_error("\\%s: missing required argument\n", cmd);
+ success = false;
+ }
+ else
+ {
+ char *fname = psql_scan_slash_option(scan_state,
+ OT_FILEPIPE, NULL, true);
+
+ if (!fname)
+ {
+ psql_error("\\%s: missing required argument\n", cmd);
+ success = false;
+ }
+ else
+ {
+ PQExpBufferData rawbuf;
+
+ initPQExpBuffer(&rawbuf);
+
+ expand_tilde(&fname);
+ canonicalize_path(fname);
+
+ if (read_file(fname, &rawbuf))
+ {
+ char *newval;
+
+ /* do bytea escaping when it is required */
+ if (strcmp(cmd, "set_from_bfile") == 0)
+ {
+ size_t escaped_size;
+
+ newval = (char *) PQescapeByteaConn(pset.db,
+ (const unsigned char *) rawbuf.data, rawbuf.len,
+ &escaped_size);
+ }
+ else
+ newval = rawbuf.data;
+
+ if (!newval)
+ {
+ psql_error("%s\n", PQerrorMessage(pset.db));
+ success = false;
+ }
+ else
+ {
+ if (!SetVariable(pset.vars, varname, newval))
+ {
+ psql_error("\\%s: error while setting variable\n", cmd);
+ success = false;
+ }
+
+ /* release Bytea escaped result */
+ if (newval != rawbuf.data)
+ PQfreemem(newval);
+ }
+ }
+ else
+ success = false;
+
+ /* release raw content */
+ termPQExpBuffer(&rawbuf);
+
+ if (fname)
+ free(fname);
+ }
+ }
+ free(varname);
+ }
/* \setenv -- set environment command */
else if (strcmp(cmd, "setenv") == 0)
@@ -3657,3 +3734,53 @@ minimal_error_message(PGresult *res)
destroyPQExpBuffer(msg);
}
+
+/*
+ * file-content-fetching callback for read file content commands.
+ */
+static bool
+read_file(char *fname, PQExpBuffer rawbuf)
+{
+ FILE *fd;
+ bool result = false;
+
+ fd = fopen(fname, PG_BINARY_R);
+ if (fd)
+ {
+ struct stat fst;
+
+ if (fstat(fileno(fd), &fst) != -1)
+ {
+ if (S_ISREG(fst.st_mode))
+ {
+ if (fst.st_size <= ((int64) 1024) * 1024 * 1024)
+ {
+ size_t size;
+ char buf[512];
+
+ while ((size = fread(buf, 1, sizeof(buf), fd)) > 0)
+ appendBinaryPQExpBuffer(rawbuf, buf, size);
+
+ if (ferror(fd))
+ psql_error("%s: %s\n", fname, strerror(errno));
+ else if (PQExpBufferBroken(rawbuf))
+ psql_error("out of memory\n");
+ else
+ result = true;
+ }
+ else
+ psql_error("%s is too big (greather than 1GB)\n", fname);
+ }
+ else
+ psql_error("%s is not regular file\n", fname);
+ }
+ else
+ psql_error("%s: %s\n", fname, strerror(errno));
+
+ fclose(fd);
+ }
+ else
+ psql_error("%s: %s\n", fname, strerror(errno));
+
+ return result;
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index cd64c39..c22046a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1339,8 +1339,9 @@ psql_completion(const char *text, int start, int end)
"\\f", "\\g", "\\gexec", "\\gset", "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
"\\lo_import", "\\lo_export", "\\lo_list", "\\lo_unlink",
"\\o", "\\p", "\\password", "\\prompt", "\\pset", "\\q", "\\qecho", "\\r",
- "\\s", "\\set", "\\setenv", "\\sf", "\\sv", "\\t", "\\T",
- "\\timing", "\\unset", "\\x", "\\w", "\\watch", "\\z", "\\!", NULL
+ "\\s", "\\set", "\\setenv", "\\set_from_bfile", "\\set_from_file",
+ "\\sf", "\\sv", "\\t", "\\T", "\\timing", "\\unset", "\\x",
+ "\\w", "\\watch", "\\z", "\\!", NULL
};
(void) end; /* "end" is not used */
@@ -3236,6 +3237,15 @@ psql_completion(const char *text, int start, int end)
else if (TailMatchesCS1("VERBOSITY"))
COMPLETE_WITH_LIST_CS3("default", "verbose", "terse");
}
+ else if (TailMatchesCS1("\\set_from_bfile|\\set_from_file"))
+ {
+ matches = complete_from_variables(text, "", "", false);
+ }
+ else if (TailMatchesCS2("\\set_from_bfile|\\set_from_file", MatchAny))
+ {
+ completion_charp = "\\";
+ matches = completion_matches(text, complete_from_files);
+ }
else if (TailMatchesCS1("\\sf*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
else if (TailMatchesCS1("\\sv*"))
diff --git a/src/test/regress/input/misc.source b/src/test/regress/input/misc.source
index dd2d1b2..eead8ae 100644
--- a/src/test/regress/input/misc.source
+++ b/src/test/regress/input/misc.source
@@ -273,3 +273,21 @@ drop table oldstyle_test;
--
-- rewrite rules
--
+
+---
+--- load file and store it to variable
+---
+CREATE TABLE test_setref(a text, b bytea);
+
+-- use two different ways for import data - result should be same
+\lo_import @abs_builddir@/data/hash.data
+\set lo_oid :LASTOID
+INSERT INTO test_setref (b) VALUES(lo_get(:lo_oid));
+\lo_unlink :lo_oid
+SELECT md5(b) FROM test_setref;
+TRUNCATE test_setref;
+
+\set_from_file var1 @abs_builddir@/data/hash.data
+\set_from_bfile var2 @abs_builddir@/data/hash.data
+INSERT INTO test_setref(a,b) VALUES(:'var1', :'var2');
+SELECT md5(a), md5(b) FROM test_setref;
diff --git a/src/test/regress/output/misc.source b/src/test/regress/output/misc.source
index 574ef0d..52c78fe 100644
--- a/src/test/regress/output/misc.source
+++ b/src/test/regress/output/misc.source
@@ -708,3 +708,28 @@ drop table oldstyle_test;
--
-- rewrite rules
--
+---
+--- load file and store it to variable
+---
+CREATE TABLE test_setref(a text, b bytea);
+-- use two different ways for import data - result should be same
+\lo_import @abs_builddir@/data/hash.data
+\set lo_oid :LASTOID
+INSERT INTO test_setref (b) VALUES(lo_get(:lo_oid));
+\lo_unlink :lo_oid
+SELECT md5(b) FROM test_setref;
+ md5
+----------------------------------
+ e446fe6ea5a347e69670633412c7f8cb
+(1 row)
+
+TRUNCATE test_setref;
+\set_from_file var1 @abs_builddir@/data/hash.data
+\set_from_bfile var2 @abs_builddir@/data/hash.data
+INSERT INTO test_setref(a,b) VALUES(:'var1', :'var2');
+SELECT md5(a), md5(b) FROM test_setref;
+ md5 | md5
+----------------------------------+----------------------------------
+ e446fe6ea5a347e69670633412c7f8cb | e446fe6ea5a347e69670633412c7f8cb
+(1 row)
+
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: tested, failed
Pavel,
gstore/gbstore:
The functionality worked as expected - one row, one column results of queries can be sent to a file or shell. It would be nice if a test case was included that proves results more than one row, one column wide will fail.
The documentation included is awkward to read. How about:
"Sends the current query input buffer to the server and stores
the result to an output file specified in the query or pipes the output
to a shell command. The file or command are written to only if the query
successfully returns exactly one, non-null row and column. If the
query fails or does not return data, an error is raised. "
Parameterized Queries:
The functionality proposed works as expected. Throughout the documentation, code and test cases the word "Parameterized" is spelled incorrectly: "PARAMETRIZED_QUERIES"
set_from_file/set_from_bfile:
The functionality proposed worked fine, I was able to set variables in sql from files. Minor typo in the documentation:
"The content is escapeaed as bytea value."
Hope this helps!
Jason O'Donnell
Crunchy Data
The new status of this patch is: Waiting on Author
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi
Thank you for review
2017-01-09 17:24 GMT+01:00 Jason O'Donnell <odonnelljp01@gmail.com>:
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: tested, failedPavel,
gstore/gbstore:
The functionality worked as expected - one row, one column results of
queries can be sent to a file or shell. It would be nice if a test case
was included that proves results more than one row, one column wide will
fail.
fixed
The documentation included is awkward to read. How about:
"Sends the current query input buffer to the server and stores
the result to an output file specified in the query or pipes the output
to a shell command. The file or command are written to only if the query
successfully returns exactly one, non-null row and column. If the
query fails or does not return data, an error is raised. "
super
Parameterized Queries:
The functionality proposed works as expected. Throughout the
documentation, code and test cases the word "Parameterized" is spelled
incorrectly: "PARAMETRIZED_QUERIES"
fixed
set_from_file/set_from_bfile:
The functionality proposed worked fine, I was able to set variables in sql
from files. Minor typo in the documentation:
"The content is escapeaed as bytea value."
fixed
Show quoted text
Hope this helps!
Jason O'Donnell
Crunchy DataThe new status of this patch is: Waiting on Author
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
psql-gstore-02.patchtext/x-patch; charset=US-ASCII; name=psql-gstore-02.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 9915731..4f95f86 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -1944,6 +1944,28 @@ hello 10
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>\gstore [ <replaceable class="parameter">filename</replaceable> ]</literal></term>
+ <term><literal>\gstore [ |<replaceable class="parameter">command</replaceable> ]</literal></term>
+ <term><literal>\gbstore [ <replaceable class="parameter">filename</replaceable> ]</literal></term>
+ <term><literal>\gbstore [ |<replaceable class="parameter">command</replaceable> ]</literal></term>
+ <listitem>
+ <para>
+ Sends the current query input buffer to the server and stores
+ the result to an output file specified in the query or pipes the output
+ to a shell command. The file or command are written to only if the query
+ successfully returns exactly one, non-null row and column. If the
+ query fails or does not return data, an error is raised.
+<programlisting>
+=> <userinput>SELECT avatar FROM users WHERE id = 123</userinput>
+-> <userinput>\gbstore ~/avatar.png</userinput>
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
+
+
<varlistentry>
<term><literal>\h</literal> or <literal>\help</literal> <literal>[ <replaceable class="parameter">command</replaceable> ]</literal></term>
<listitem>
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 4139b77..33f4559 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -929,6 +929,27 @@ exec_command(const char *cmd,
status = PSQL_CMD_SEND;
}
+ /* \gstore [filename], \gbstore [filename] -- send query and store result in (binary) file */
+ else if (strcmp(cmd, "gstore") == 0 ||
+ (strcmp(cmd, "gbstore") == 0))
+ {
+ char *fname = psql_scan_slash_option(scan_state,
+ OT_FILEPIPE, NULL, false);
+
+ if (!fname)
+ pset.gfname = pg_strdup("");
+ else
+ {
+ expand_tilde(&fname);
+ pset.gfname = pg_strdup(fname);
+ }
+
+ pset.raw_flag = true;
+ pset.binres_flag = (strcmp(cmd, "gbstore") == 0);
+ free(fname);
+ status = PSQL_CMD_SEND;
+ }
+
/* help */
else if (strcmp(cmd, "h") == 0 || strcmp(cmd, "help") == 0)
{
@@ -1064,7 +1085,6 @@ exec_command(const char *cmd,
free(opt2);
}
-
/* \o -- set query output */
else if (strcmp(cmd, "o") == 0 || strcmp(cmd, "out") == 0)
{
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index e1b04de..a6aaebe 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -854,6 +854,85 @@ StoreQueryTuple(const PGresult *result)
return success;
}
+/*
+ * StoreRawResult: the returned value (possibly binary) is displayed
+ * or stored in file. The result should be exactly one row, one column.
+ */
+static bool
+StoreRawResult(const PGresult *result)
+{
+ bool success = true;
+
+ if (PQntuples(result) < 1)
+ {
+ psql_error("no rows returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQntuples(result) > 1)
+ {
+ psql_error("more than one row returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQnfields(result) < 1)
+ {
+ psql_error("no columns returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQnfields(result) > 1)
+ {
+ psql_error("more than one column returned for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else if (PQgetisnull(result, 0, 0))
+ {
+ psql_error("returned value is null for \\gstore or \\gbstore\n");
+ success = false;
+ }
+ else
+ {
+ char *value;
+ int length;
+ FILE *fout = NULL;
+ bool is_pipe = false;
+
+ value = PQgetvalue(result, 0, 0);
+ length = PQgetlength(result, 0, 0);
+
+ if (pset.gfname && *(pset.gfname) != '\0')
+ {
+ if (!openQueryOutputFile(pset.gfname, &fout, &is_pipe))
+ success = false;
+ if (success && is_pipe)
+ disable_sigpipe_trap();
+ }
+
+ if (success)
+ {
+ success = fwrite(value, 1, length, fout != NULL ? fout : pset.queryFout) == length;
+ if (!success)
+ psql_error("%s: %s\n", pset.gfname, strerror(errno));
+
+ if (success)
+ success = fflush(fout != NULL ? fout : pset.queryFout) == 0;
+
+ if (!success)
+ psql_error("%s: %s\n", pset.gfname, strerror(errno));
+
+ if (fout != NULL)
+ {
+ if (is_pipe)
+ {
+ pclose(fout);
+ restore_sigpipe_trap();
+ }
+ else
+ fclose(fout);
+ }
+ }
+ }
+
+ return success;
+}
/*
* ExecQueryTuples: assuming query result is OK, execute each query
@@ -1124,6 +1203,8 @@ PrintQueryResults(PGresult *results)
success = ExecQueryTuples(results);
else if (pset.crosstab_flag)
success = PrintResultsInCrosstab(results);
+ else if (pset.raw_flag)
+ success = StoreRawResult(results);
else
success = PrintQueryTuples(results);
/* if it's INSERT/UPDATE/DELETE RETURNING, also print status */
@@ -1278,7 +1359,8 @@ SendQuery(const char *query)
}
if (pset.fetch_count <= 0 || pset.gexec_flag ||
- pset.crosstab_flag || !is_select_command(query))
+ pset.crosstab_flag || !is_select_command(query) ||
+ pset.raw_flag)
{
/* Default fetch-it-all-and-print mode */
instr_time before,
@@ -1287,7 +1369,16 @@ SendQuery(const char *query)
if (pset.timing)
INSTR_TIME_SET_CURRENT(before);
- results = PQexec(pset.db, query);
+ if (pset.binres_flag)
+ results = PQexecParams(pset.db, query,
+ 0,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ pset.binres_flag);
+ else
+ results = PQexec(pset.db, query);
/* these operations are included in the timing result: */
ResetCancelConn();
@@ -1404,7 +1495,7 @@ SendQuery(const char *query)
sendquery_cleanup:
- /* reset \g's output-to-filename trigger */
+ /* reset \g, \g[b]store output-to-filename trigger */
if (pset.gfname)
{
free(pset.gfname);
@@ -1421,6 +1512,10 @@ sendquery_cleanup:
/* reset \gexec trigger */
pset.gexec_flag = false;
+ /* reset \gstore, gbstore trigger */
+ pset.raw_flag = false;
+ pset.binres_flag = false;
+
/* reset \crosstabview trigger */
pset.crosstab_flag = false;
for (i = 0; i < lengthof(pset.ctv_args); i++)
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index 09baf87..be26ff0 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -168,7 +168,7 @@ slashUsage(unsigned short int pager)
* Use "psql --help=commands | wc" to count correctly. It's okay to count
* the USE_READLINE line even in builds without that.
*/
- output = PageOutput(113, pager ? &(pset.popt.topt) : NULL);
+ output = PageOutput(115, pager ? &(pset.popt.topt) : NULL);
fprintf(output, _("General\n"));
fprintf(output, _(" \\copyright show PostgreSQL usage and distribution terms\n"));
@@ -176,6 +176,8 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\g [FILE] or ; execute query (and send results to file or |pipe)\n"));
fprintf(output, _(" \\gexec execute query, then execute each value in its result\n"));
fprintf(output, _(" \\gset [PREFIX] execute query and store results in psql variables\n"));
+ fprintf(output, _(" \\gstore [FILE] execute query and store result to file or |pipe\n"));
+ fprintf(output, _(" \\gbstore [FILE] execute query and store bin result to file or |pipe\n"));
fprintf(output, _(" \\q quit psql\n"));
fprintf(output, _(" \\crosstabview [COLUMNS] execute query and display results in crosstab\n"));
fprintf(output, _(" \\watch [SEC] execute query every SEC seconds\n"));
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 4c7c3b1..1c5a68d 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -95,6 +95,8 @@ typedef struct _psqlSettings
bool gexec_flag; /* one-shot flag to execute query's results */
bool crosstab_flag; /* one-shot request to crosstab results */
char *ctv_args[4]; /* \crosstabview arguments */
+ bool raw_flag; /* one-shot flag to work with exact one value */
+ bool binres_flag; /* one-shot flag - enforce binary result format */
bool notty; /* stdin or stdout is not a tty (as determined
* on startup) */
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 7709112..36e5c1a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1336,7 +1336,8 @@ psql_completion(const char *text, int start, int end)
"\\dm", "\\dn", "\\do", "\\dO", "\\dp", "\\drds", "\\ds", "\\dS",
"\\dt", "\\dT", "\\dv", "\\du", "\\dx", "\\dy",
"\\e", "\\echo", "\\ef", "\\encoding", "\\errverbose", "\\ev",
- "\\f", "\\g", "\\gexec", "\\gset", "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
+ "\\f", "\\g", "\\gbstore", "\\gexec", "\\gset", "gstore",
+ "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
"\\lo_import", "\\lo_export", "\\lo_list", "\\lo_unlink",
"\\o", "\\p", "\\password", "\\prompt", "\\pset", "\\q", "\\qecho", "\\r",
"\\s", "\\set", "\\setenv", "\\sf", "\\sv", "\\t", "\\T",
@@ -3279,8 +3280,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
else if (TailMatchesCS1("\\sv*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
- else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\i|\\include|"
- "\\ir|\\include_relative|\\o|\\out|"
+ else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\gbstore|\\gstore|"
+ "\\i|\\include|\\ir|\\include_relative|\\o|\\out|"
"\\s|\\w|\\write|\\lo_import"))
{
completion_charp = "\\";
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 464436a..b2aedbe 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2714,3 +2714,24 @@ NOTICE: foo
CONTEXT: PL/pgSQL function inline_code_block line 3 at RAISE
ERROR: bar
CONTEXT: PL/pgSQL function inline_code_block line 4 at RAISE
+-- should fail
+select 'Hello','Hello'
+\gstore foofile
+more than one column returned for \gstore or \gbstore
+select E'\\xDEADBEEF'::bytea, E'\\xDEADBEEF'::bytea
+\gbstore foofile
+more than one column returned for \gstore or \gbstore
+select 'hello' union all select 'hello'
+\gstore foofile
+more than one row returned for \gstore or \gbstore
+select E'\\xDEADBEEF'::bytea union all E'\\xDEADBEEF'::bytea
+\gbstore foofile
+ERROR: syntax error at or near "E'\\xDEADBEEF'"
+LINE 1: select E'\\xDEADBEEF'::bytea union all E'\\xDEADBEEF'::bytea
+ ^
+select 'hello' where false
+\gstore foofile
+no rows returned for \gstore or \gbstore
+select E'\\xDEADBEEF'::bytea where false
+\gbstore foofile
+no rows returned for \gstore or \gbstore
diff --git a/src/test/regress/input/misc.source b/src/test/regress/input/misc.source
index dd2d1b2..39cde61 100644
--- a/src/test/regress/input/misc.source
+++ b/src/test/regress/input/misc.source
@@ -273,3 +273,28 @@ drop table oldstyle_test;
--
-- rewrite rules
--
+
+--
+-- psql gstore, gbstore commands
+--
+CREATE TABLE test_store(a text, b bytea);
+INSERT INTO test_store values('AHOJ', E'\\xDEADBEEF');
+SELECT md5(a) a, md5(b) b FROM test_store;
+
+SELECT a FROM test_store
+\gstore @abs_builddir@/data/test_store.txt
+SELECT b FROM test_store
+\gbstore @abs_builddir@/data/test_store.bin
+
+\lo_import @abs_builddir@/data/test_store.txt
+\set lo_oid :LASTOID
+SELECT md5(lo_get(:lo_oid));
+\lo_unlink :lo_oid
+
+\lo_import @abs_builddir@/data/test_store.bin
+\set lo_oid :LASTOID
+SELECT md5(lo_get(:lo_oid));
+\lo_unlink :lo_oid
+
+DROP TABLE test_store;
+
diff --git a/src/test/regress/output/misc.source b/src/test/regress/output/misc.source
index 574ef0d..eef17ec 100644
--- a/src/test/regress/output/misc.source
+++ b/src/test/regress/output/misc.source
@@ -708,3 +708,37 @@ drop table oldstyle_test;
--
-- rewrite rules
--
+--
+-- psql gstore, gbstore commands
+--
+CREATE TABLE test_store(a text, b bytea);
+INSERT INTO test_store values('AHOJ', E'\\xDEADBEEF');
+SELECT md5(a) a, md5(b) b FROM test_store;
+ a | b
+----------------------------------+----------------------------------
+ 5d75193725cfb92ce9aee96b5380db06 | 2f249230a8e7c2bf6005ccd2679259ec
+(1 row)
+
+SELECT a FROM test_store
+\gstore @abs_builddir@/data/test_store.txt
+SELECT b FROM test_store
+\gbstore @abs_builddir@/data/test_store.bin
+\lo_import @abs_builddir@/data/test_store.txt
+\set lo_oid :LASTOID
+SELECT md5(lo_get(:lo_oid));
+ md5
+----------------------------------
+ 5d75193725cfb92ce9aee96b5380db06
+(1 row)
+
+\lo_unlink :lo_oid
+\lo_import @abs_builddir@/data/test_store.bin
+\set lo_oid :LASTOID
+SELECT md5(lo_get(:lo_oid));
+ md5
+----------------------------------
+ 2f249230a8e7c2bf6005ccd2679259ec
+(1 row)
+
+\lo_unlink :lo_oid
+DROP TABLE test_store;
diff --git a/src/test/regress/sql/psql.sql b/src/test/regress/sql/psql.sql
index 900aa7e..7edc1e8 100644
--- a/src/test/regress/sql/psql.sql
+++ b/src/test/regress/sql/psql.sql
@@ -379,3 +379,22 @@ begin
raise notice 'foo';
raise exception 'bar';
end $$;
+
+-- should fail
+select 'Hello','Hello'
+\gstore foofile
+
+select E'\\xDEADBEEF'::bytea, E'\\xDEADBEEF'::bytea
+\gbstore foofile
+
+select 'hello' union all select 'hello'
+\gstore foofile
+
+select E'\\xDEADBEEF'::bytea union all E'\\xDEADBEEF'::bytea
+\gbstore foofile
+
+select 'hello' where false
+\gstore foofile
+
+select E'\\xDEADBEEF'::bytea where false
+\gbstore foofile
psql-parameterized-queries-02.patchtext/x-patch; charset=US-ASCII; name=psql-parameterized-queries-02.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 9915731..e47e8d5 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3353,6 +3353,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>PARAMETERIZED_QUERIES</varname></term>
+ <listitem>
+ <para>
+ The psql's variables can be injected to query text (by default) or
+ passed as query parameters when this variable is set
+ <literal>on</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>PORT</varname></term>
<listitem>
<para>
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index e1b04de..94c2993 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -119,9 +119,13 @@ setQFout(const char *fname)
* If "escape" is true, return the value suitably quoted and escaped,
* as an identifier or string literal depending on "as_ident".
* (Failure in escaping should lead to returning NULL.)
+ *
+ * When "inside_query" is true, then the variable can be passed as query parameter,
+ * when it is not used as identifier (as_ident:false), when escape is not required
+ * (escaping changes the content).
*/
char *
-psql_get_variable(const char *varname, bool escape, bool as_ident)
+psql_get_variable(const char *varname, bool escape, bool as_ident, bool inside_query)
{
char *result;
const char *value;
@@ -130,6 +134,35 @@ psql_get_variable(const char *varname, bool escape, bool as_ident)
if (!value)
return NULL;
+ if (inside_query && pset.parameterized_queries)
+ {
+ if (!escape && !as_ident)
+ {
+ char printbuf[10];
+
+ if (pset.nparams >= pset.max_params)
+ {
+ /* create or realloc params array */
+ if (pset.max_params > 0)
+ {
+ pset.max_params += 16;
+ pset.params = (const char **) pg_realloc(pset.params,
+ sizeof(const char *) * pset.max_params);
+ }
+ else
+ {
+ pset.max_params = 16;
+ pset.params = (const char **) pg_malloc(sizeof(const char *) * pset.max_params);
+ }
+ }
+
+ pset.params[pset.nparams++] = value;
+ snprintf(printbuf, sizeof(printbuf) - 1, "$%d", pset.nparams);
+
+ return pstrdup(printbuf);
+ }
+ }
+
if (escape)
{
char *escaped_value;
@@ -1287,7 +1320,16 @@ SendQuery(const char *query)
if (pset.timing)
INSTR_TIME_SET_CURRENT(before);
- results = PQexec(pset.db, query);
+ if (pset.nparams > 0)
+ results = PQexecParams(pset.db, query,
+ pset.nparams,
+ NULL,
+ (const char * const *) pset.params,
+ NULL,
+ NULL,
+ 0);
+ else
+ results = PQexec(pset.db, query);
/* these operations are included in the timing result: */
ResetCancelConn();
@@ -1382,6 +1424,15 @@ SendQuery(const char *query)
ClearOrSaveResult(results);
+ /* the number of query parameters are not necessary now */
+ pset.nparams = 0;
+ if (pset.max_params > 0)
+ {
+ free(pset.params);
+ pset.params = NULL;
+ pset.max_params = 0;
+ }
+
/* Possible microtiming output */
if (pset.timing)
PrintTiming(elapsed_msec);
@@ -1488,7 +1539,16 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
appendPQExpBuffer(&buf, "DECLARE _psql_cursor NO SCROLL CURSOR FOR\n%s",
query);
- results = PQexec(pset.db, buf.data);
+ if (pset.nparams > 0)
+ results = PQexecParams(pset.db, buf.data,
+ pset.nparams,
+ NULL,
+ (const char * const *) pset.params,
+ NULL,
+ NULL,
+ 0);
+ else
+ results = PQexec(pset.db, buf.data);
OK = AcceptResult(results) &&
(PQresultStatus(results) == PGRES_COMMAND_OK);
ClearOrSaveResult(results);
diff --git a/src/bin/psql/common.h b/src/bin/psql/common.h
index dad0eb8..4dba1e1 100644
--- a/src/bin/psql/common.h
+++ b/src/bin/psql/common.h
@@ -18,7 +18,7 @@
extern bool openQueryOutputFile(const char *fname, FILE **fout, bool *is_pipe);
extern bool setQFout(const char *fname);
-extern char *psql_get_variable(const char *varname, bool escape, bool as_ident);
+extern char *psql_get_variable(const char *varname, bool escape, bool as_ident, bool inside_query);
extern void psql_error(const char *fmt,...) pg_attribute_printf(1, 2);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index 09baf87..1f4ab2c 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -325,7 +325,7 @@ helpVariables(unsigned short int pager)
* Windows builds currently print one more line than non-Windows builds.
* Using the larger number is fine.
*/
- output = PageOutput(88, pager ? &(pset.popt.topt) : NULL);
+ output = PageOutput(90, pager ? &(pset.popt.topt) : NULL);
fprintf(output, _("List of specially treated variables\n\n"));
@@ -352,6 +352,8 @@ helpVariables(unsigned short int pager)
fprintf(output, _(" LASTOID value of the last affected OID\n"));
fprintf(output, _(" ON_ERROR_ROLLBACK if set, an error doesn't stop a transaction (uses implicit savepoints)\n"));
fprintf(output, _(" ON_ERROR_STOP stop batch execution after error\n"));
+ fprintf(output, _(" PARAMETERIZED_QUERIES\n"
+ " pass psql's variables as query parameters\n"));
fprintf(output, _(" PORT server port of the current connection\n"));
fprintf(output, _(" PROMPT1 specifies the standard psql prompt\n"));
fprintf(output, _(" PROMPT2 specifies the prompt used when a statement continues from a previous line\n"));
diff --git a/src/bin/psql/mainloop.c b/src/bin/psql/mainloop.c
index bb306a4..c4828fe 100644
--- a/src/bin/psql/mainloop.c
+++ b/src/bin/psql/mainloop.c
@@ -403,6 +403,15 @@ MainLoop(FILE *source)
psql_scan_finish(scan_state);
free(line);
+ /* reset a number of query parameters */
+ pset.nparams = 0;
+ if (pset.max_params > 0)
+ {
+ free(pset.params);
+ pset.params = NULL;
+ pset.max_params = 0;
+ }
+
if (slashCmdStatus == PSQL_CMD_TERMINATE)
{
successResult = EXIT_SUCCESS;
diff --git a/src/bin/psql/psqlscanslash.l b/src/bin/psql/psqlscanslash.l
index 5b7953b..3e58303 100644
--- a/src/bin/psql/psqlscanslash.l
+++ b/src/bin/psql/psqlscanslash.l
@@ -243,6 +243,7 @@ other .
yyleng - 1);
value = cur_state->callbacks->get_variable(varname,
false,
+ false,
false);
free(varname);
@@ -271,7 +272,7 @@ other .
ECHO;
else
{
- psqlscan_escape_variable(cur_state, yytext, yyleng, false);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, false, false);
*option_quote = ':';
}
unquoted_option_chars = 0;
@@ -283,7 +284,7 @@ other .
ECHO;
else
{
- psqlscan_escape_variable(cur_state, yytext, yyleng, true);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, true, false);
*option_quote = ':';
}
unquoted_option_chars = 0;
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 4c7c3b1..7d21e46 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -120,6 +120,7 @@ typedef struct _psqlSettings
* functions.
*/
bool autocommit;
+ bool parameterized_queries;
bool on_error_stop;
bool quiet;
bool singleline;
@@ -135,6 +136,9 @@ typedef struct _psqlSettings
const char *prompt3;
PGVerbosity verbosity; /* current error verbosity level */
PGContextVisibility show_context; /* current context display level */
+ int nparams; /* number of query parameters */
+ int max_params; /* max size of current parameters array */
+ const char **params; /* query parameters */
} PsqlSettings;
extern PsqlSettings pset;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 85aac4a..441a0ca 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -139,6 +139,10 @@ main(int argc, char *argv[])
pset.last_error_result = NULL;
pset.cur_cmd_source = stdin;
pset.cur_cmd_interactive = false;
+ pset.parameterized_queries = false;
+ pset.nparams = 0;
+ pset.max_params = 0;
+ pset.params = NULL;
/* We rely on unmentioned fields of pset.popt to start out 0/false/NULL */
pset.popt.topt.format = PRINT_ALIGNED;
@@ -793,6 +797,12 @@ autocommit_hook(const char *newval)
}
static void
+parameterized_queries_hook(const char *newval)
+{
+ pset.parameterized_queries = ParseVariableBool(newval, "PARAMETERIZED_QUERIES");
+}
+
+static void
on_error_stop_hook(const char *newval)
{
pset.on_error_stop = ParseVariableBool(newval, "ON_ERROR_STOP");
@@ -990,6 +1000,7 @@ EstablishVariableSpace(void)
SetVariableAssignHook(pset.vars, "ON_ERROR_ROLLBACK", on_error_rollback_hook);
SetVariableAssignHook(pset.vars, "COMP_KEYWORD_CASE", comp_keyword_case_hook);
SetVariableAssignHook(pset.vars, "HISTCONTROL", histcontrol_hook);
+ SetVariableAssignHook(pset.vars, "PARAMETERIZED_QUERIES", parameterized_queries_hook);
SetVariableAssignHook(pset.vars, "PROMPT1", prompt1_hook);
SetVariableAssignHook(pset.vars, "PROMPT2", prompt2_hook);
SetVariableAssignHook(pset.vars, "PROMPT3", prompt3_hook);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 7709112..6f614df 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3255,8 +3255,8 @@ psql_completion(const char *text, int start, int end)
}
else if (TailMatchesCS2("\\set", MatchAny))
{
- if (TailMatchesCS1("AUTOCOMMIT|ON_ERROR_STOP|QUIET|"
- "SINGLELINE|SINGLESTEP"))
+ if (TailMatchesCS1("AUTOCOMMIT|ON_ERROR_STOP|PARAMETERIZED_QUERIES|"
+ "QUIET|SINGLELINE|SINGLESTEP"))
COMPLETE_WITH_LIST_CS2("on", "off");
else if (TailMatchesCS1("COMP_KEYWORD_CASE"))
COMPLETE_WITH_LIST_CS4("lower", "upper",
@@ -3748,7 +3748,7 @@ complete_from_variables(const char *text, const char *prefix, const char *suffix
"AUTOCOMMIT", "COMP_KEYWORD_CASE", "DBNAME", "ECHO", "ECHO_HIDDEN",
"ENCODING", "FETCH_COUNT", "HISTCONTROL", "HISTFILE", "HISTSIZE",
"HOST", "IGNOREEOF", "LASTOID", "ON_ERROR_ROLLBACK", "ON_ERROR_STOP",
- "PORT", "PROMPT1", "PROMPT2", "PROMPT3", "QUIET",
+ "PARAMETERIZED_QUERIES", "PORT", "PROMPT1", "PROMPT2", "PROMPT3", "QUIET",
"SHOW_CONTEXT", "SINGLELINE", "SINGLESTEP",
"USER", "VERBOSITY", NULL
};
diff --git a/src/fe_utils/psqlscan.l b/src/fe_utils/psqlscan.l
index 1b29341..6ab9c0b 100644
--- a/src/fe_utils/psqlscan.l
+++ b/src/fe_utils/psqlscan.l
@@ -700,7 +700,8 @@ other .
if (cur_state->callbacks->get_variable)
value = cur_state->callbacks->get_variable(varname,
false,
- false);
+ false,
+ true);
else
value = NULL;
@@ -736,11 +737,11 @@ other .
}
:'{variable_char}+' {
- psqlscan_escape_variable(cur_state, yytext, yyleng, false);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, false, true);
}
:\"{variable_char}+\" {
- psqlscan_escape_variable(cur_state, yytext, yyleng, true);
+ psqlscan_escape_variable(cur_state, yytext, yyleng, true, true);
}
/*
@@ -1401,7 +1402,7 @@ psqlscan_extract_substring(PsqlScanState state, const char *txt, int len)
*/
void
psqlscan_escape_variable(PsqlScanState state, const char *txt, int len,
- bool as_ident)
+ bool as_ident, bool from_query)
{
char *varname;
char *value;
@@ -1409,7 +1410,8 @@ psqlscan_escape_variable(PsqlScanState state, const char *txt, int len,
/* Variable lookup. */
varname = psqlscan_extract_substring(state, txt + 2, len - 3);
if (state->callbacks->get_variable)
- value = state->callbacks->get_variable(varname, true, as_ident);
+ value = state->callbacks->get_variable(varname,
+ true, as_ident, from_query);
else
value = NULL;
free(varname);
diff --git a/src/include/fe_utils/psqlscan.h b/src/include/fe_utils/psqlscan.h
index 21c4f22..e14764d 100644
--- a/src/include/fe_utils/psqlscan.h
+++ b/src/include/fe_utils/psqlscan.h
@@ -53,7 +53,8 @@ typedef struct PsqlScanCallbacks
{
/* Fetch value of a variable, as a pfree'able string; NULL if unknown */
/* This pointer can be NULL if no variable substitution is wanted */
- char *(*get_variable) (const char *varname, bool escape, bool as_ident);
+ char *(*get_variable) (const char *varname,
+ bool escape, bool as_ident, bool from_query);
/* Print an error message someplace appropriate */
/* (very old gcc versions don't support attributes on function pointers) */
#if defined(__GNUC__) && __GNUC__ < 4
diff --git a/src/include/fe_utils/psqlscan_int.h b/src/include/fe_utils/psqlscan_int.h
index 0fddc7a..1fb2793 100644
--- a/src/include/fe_utils/psqlscan_int.h
+++ b/src/include/fe_utils/psqlscan_int.h
@@ -139,6 +139,7 @@ extern char *psqlscan_extract_substring(PsqlScanState state,
const char *txt, int len);
extern void psqlscan_escape_variable(PsqlScanState state,
const char *txt, int len,
- bool as_ident);
+ bool as_ident,
+ bool from_query);
#endif /* PSQLSCAN_INT_H */
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 464436a..4f3c580 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2714,3 +2714,36 @@ NOTICE: foo
CONTEXT: PL/pgSQL function inline_code_block line 3 at RAISE
ERROR: bar
CONTEXT: PL/pgSQL function inline_code_block line 4 at RAISE
+-- parameterized queries
+\set PARAMETERIZED_QUERIES off
+\set a1 'AHOJ SVETE'
+-- should fail
+SELECT :a1;
+ERROR: column "ahoj" does not exist
+LINE 1: SELECT AHOJ SVETE;
+ ^
+-- ok
+SELECT :'a1';
+ ?column?
+------------
+ AHOJ SVETE
+(1 row)
+
+\set PARAMETERIZED_QUERIES on
+-- should fail - unknown type
+SELECT :a1;
+ERROR: could not determine data type of parameter $1
+-- ok
+SELECT :a1::text;
+ text
+------------
+ AHOJ SVETE
+(1 row)
+
+-- returns true, when value passed as parameter is same as client side evaluated variable
+SELECT :a1 = :'a1';
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/sql/psql.sql b/src/test/regress/sql/psql.sql
index 900aa7e..fbc9302 100644
--- a/src/test/regress/sql/psql.sql
+++ b/src/test/regress/sql/psql.sql
@@ -379,3 +379,23 @@ begin
raise notice 'foo';
raise exception 'bar';
end $$;
+
+-- parameterized queries
+\set PARAMETERIZED_QUERIES off
+\set a1 'AHOJ SVETE'
+
+-- should fail
+SELECT :a1;
+
+-- ok
+SELECT :'a1';
+
+\set PARAMETERIZED_QUERIES on
+-- should fail - unknown type
+SELECT :a1;
+
+-- ok
+SELECT :a1::text;
+
+-- returns true, when value passed as parameter is same as client side evaluated variable
+SELECT :a1 = :'a1';
psql-set-from-file-02.patchtext/x-patch; charset=US-ASCII; name=psql-set-from-file-02.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 9915731..1d77569 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -2697,6 +2697,33 @@ lo_import 152801
<varlistentry>
+ <term><literal>\set_from_bfile <replaceable class="parameter">name</replaceable> <replaceable class="parameter">filename</replaceable></literal></term>
+
+ <listitem>
+ <para>
+ Sets the <application>psql</> variable <replaceable
+ class="parameter">name</replaceable> by content of
+ binary file <replaceable class="parameter">filename</replaceable>.
+ The content is escaped as bytea value.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term><literal>\set_from_file <replaceable class="parameter">name</replaceable> <replaceable class="parameter">filename</replaceable></literal></term>
+
+ <listitem>
+ <para>
+ Sets the <application>psql</> variable <replaceable
+ class="parameter">name</replaceable> by content of
+ text file <replaceable class="parameter">filename</replaceable>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
<term><literal>\setenv <replaceable class="parameter">name</replaceable> [ <replaceable class="parameter">value</replaceable> ]</literal></term>
<listitem>
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a9a2fdb..bcc1793 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -82,6 +82,7 @@ static void minimal_error_message(PGresult *res);
static void printSSLInfo(void);
static bool printPsetInfo(const char *param, struct printQueryOpt *popt);
static char *pset_value_string(const char *param, struct printQueryOpt *popt);
+static bool read_file(char *fname, PQExpBuffer rawbuf);
#ifdef WIN32
static void checkWin32Codepage(void);
@@ -1318,6 +1319,82 @@ exec_command(const char *cmd,
free(opt0);
}
+ /* \set_from_file, \set_from_bfile -- set variable/option command from file */
+ else if (strcmp(cmd, "set_from_file") == 0 || strcmp(cmd, "set_from_bfile") == 0)
+ {
+ char *varname = psql_scan_slash_option(scan_state,
+ OT_NORMAL, NULL, false);
+
+ if (!varname)
+ {
+ psql_error("\\%s: missing required argument\n", cmd);
+ success = false;
+ }
+ else
+ {
+ char *fname = psql_scan_slash_option(scan_state,
+ OT_FILEPIPE, NULL, true);
+
+ if (!fname)
+ {
+ psql_error("\\%s: missing required argument\n", cmd);
+ success = false;
+ }
+ else
+ {
+ PQExpBufferData rawbuf;
+
+ initPQExpBuffer(&rawbuf);
+
+ expand_tilde(&fname);
+ canonicalize_path(fname);
+
+ if (read_file(fname, &rawbuf))
+ {
+ char *newval;
+
+ /* do bytea escaping when it is required */
+ if (strcmp(cmd, "set_from_bfile") == 0)
+ {
+ size_t escaped_size;
+
+ newval = (char *) PQescapeByteaConn(pset.db,
+ (const unsigned char *) rawbuf.data, rawbuf.len,
+ &escaped_size);
+ }
+ else
+ newval = rawbuf.data;
+
+ if (!newval)
+ {
+ psql_error("%s\n", PQerrorMessage(pset.db));
+ success = false;
+ }
+ else
+ {
+ if (!SetVariable(pset.vars, varname, newval))
+ {
+ psql_error("\\%s: error while setting variable\n", cmd);
+ success = false;
+ }
+
+ /* release Bytea escaped result */
+ if (newval != rawbuf.data)
+ PQfreemem(newval);
+ }
+ }
+ else
+ success = false;
+
+ /* release raw content */
+ termPQExpBuffer(&rawbuf);
+
+ if (fname)
+ free(fname);
+ }
+ }
+ free(varname);
+ }
/* \setenv -- set environment command */
else if (strcmp(cmd, "setenv") == 0)
@@ -3657,3 +3734,53 @@ minimal_error_message(PGresult *res)
destroyPQExpBuffer(msg);
}
+
+/*
+ * file-content-fetching callback for read file content commands.
+ */
+static bool
+read_file(char *fname, PQExpBuffer rawbuf)
+{
+ FILE *fd;
+ bool result = false;
+
+ fd = fopen(fname, PG_BINARY_R);
+ if (fd)
+ {
+ struct stat fst;
+
+ if (fstat(fileno(fd), &fst) != -1)
+ {
+ if (S_ISREG(fst.st_mode))
+ {
+ if (fst.st_size <= ((int64) 1024) * 1024 * 1024)
+ {
+ size_t size;
+ char buf[512];
+
+ while ((size = fread(buf, 1, sizeof(buf), fd)) > 0)
+ appendBinaryPQExpBuffer(rawbuf, buf, size);
+
+ if (ferror(fd))
+ psql_error("%s: %s\n", fname, strerror(errno));
+ else if (PQExpBufferBroken(rawbuf))
+ psql_error("out of memory\n");
+ else
+ result = true;
+ }
+ else
+ psql_error("%s is too big (greather than 1GB)\n", fname);
+ }
+ else
+ psql_error("%s is not regular file\n", fname);
+ }
+ else
+ psql_error("%s: %s\n", fname, strerror(errno));
+
+ fclose(fd);
+ }
+ else
+ psql_error("%s: %s\n", fname, strerror(errno));
+
+ return result;
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index cd64c39..c22046a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1339,8 +1339,9 @@ psql_completion(const char *text, int start, int end)
"\\f", "\\g", "\\gexec", "\\gset", "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
"\\lo_import", "\\lo_export", "\\lo_list", "\\lo_unlink",
"\\o", "\\p", "\\password", "\\prompt", "\\pset", "\\q", "\\qecho", "\\r",
- "\\s", "\\set", "\\setenv", "\\sf", "\\sv", "\\t", "\\T",
- "\\timing", "\\unset", "\\x", "\\w", "\\watch", "\\z", "\\!", NULL
+ "\\s", "\\set", "\\setenv", "\\set_from_bfile", "\\set_from_file",
+ "\\sf", "\\sv", "\\t", "\\T", "\\timing", "\\unset", "\\x",
+ "\\w", "\\watch", "\\z", "\\!", NULL
};
(void) end; /* "end" is not used */
@@ -3236,6 +3237,15 @@ psql_completion(const char *text, int start, int end)
else if (TailMatchesCS1("VERBOSITY"))
COMPLETE_WITH_LIST_CS3("default", "verbose", "terse");
}
+ else if (TailMatchesCS1("\\set_from_bfile|\\set_from_file"))
+ {
+ matches = complete_from_variables(text, "", "", false);
+ }
+ else if (TailMatchesCS2("\\set_from_bfile|\\set_from_file", MatchAny))
+ {
+ completion_charp = "\\";
+ matches = completion_matches(text, complete_from_files);
+ }
else if (TailMatchesCS1("\\sf*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
else if (TailMatchesCS1("\\sv*"))
diff --git a/src/test/regress/input/misc.source b/src/test/regress/input/misc.source
index dd2d1b2..eead8ae 100644
--- a/src/test/regress/input/misc.source
+++ b/src/test/regress/input/misc.source
@@ -273,3 +273,21 @@ drop table oldstyle_test;
--
-- rewrite rules
--
+
+---
+--- load file and store it to variable
+---
+CREATE TABLE test_setref(a text, b bytea);
+
+-- use two different ways for import data - result should be same
+\lo_import @abs_builddir@/data/hash.data
+\set lo_oid :LASTOID
+INSERT INTO test_setref (b) VALUES(lo_get(:lo_oid));
+\lo_unlink :lo_oid
+SELECT md5(b) FROM test_setref;
+TRUNCATE test_setref;
+
+\set_from_file var1 @abs_builddir@/data/hash.data
+\set_from_bfile var2 @abs_builddir@/data/hash.data
+INSERT INTO test_setref(a,b) VALUES(:'var1', :'var2');
+SELECT md5(a), md5(b) FROM test_setref;
diff --git a/src/test/regress/output/misc.source b/src/test/regress/output/misc.source
index 574ef0d..52c78fe 100644
--- a/src/test/regress/output/misc.source
+++ b/src/test/regress/output/misc.source
@@ -708,3 +708,28 @@ drop table oldstyle_test;
--
-- rewrite rules
--
+---
+--- load file and store it to variable
+---
+CREATE TABLE test_setref(a text, b bytea);
+-- use two different ways for import data - result should be same
+\lo_import @abs_builddir@/data/hash.data
+\set lo_oid :LASTOID
+INSERT INTO test_setref (b) VALUES(lo_get(:lo_oid));
+\lo_unlink :lo_oid
+SELECT md5(b) FROM test_setref;
+ md5
+----------------------------------
+ e446fe6ea5a347e69670633412c7f8cb
+(1 row)
+
+TRUNCATE test_setref;
+\set_from_file var1 @abs_builddir@/data/hash.data
+\set_from_bfile var2 @abs_builddir@/data/hash.data
+INSERT INTO test_setref(a,b) VALUES(:'var1', :'var2');
+SELECT md5(a), md5(b) FROM test_setref;
+ md5 | md5
+----------------------------------+----------------------------------
+ e446fe6ea5a347e69670633412c7f8cb | e446fe6ea5a347e69670633412c7f8cb
+(1 row)
+
On Wed, Jan 11, 2017 at 12:32 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
Thank you for review
Moved to next CF 2017-03.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Pavel,
I started looking through this to see if it might be ready to commit and
I don't believe it is. Below are my comments about the first patch, I
didn't get to the point of looking at the others yet since this one had
issues.
* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-01-09 17:24 GMT+01:00 Jason O'Donnell <odonnelljp01@gmail.com>:
gstore/gbstore:
I don't see the point to 'gstore'- how is that usefully different from
just using '\g'? Also, the comments around these are inconsistent, some
say they can only be used with a filename, others say it could be a
filename or a pipe+command.
There's a whitespace-only hunk that shouldn't be included.
I don't agree with the single-column/single-row restriction on these. I
can certainly see a case where someone might want to, say, dump out a
bunch of binary integers into a file for later processing.
The tab-completion for 'gstore' wasn't correct (you didn't include the
double-backslash). The patch also has conflicts against current master
now.
I guess my thinking about moving this forward would be to simplify it to
just '\gb' which will pull the data from the server side in binary
format and dump it out to the filename or command given. If there's a
new patch with those changes, I'll try to find time to look at it.
I would recommend going through a detailed review of the other patches
also before rebasing and re-submitting them also, in particular look to
make sure that the comments are correct and consistent, that there are
comments where there should be (generally speaking, whole functions
should have at least some comments in them, not just the function header
comment, etc).
Lastly, I'd suggest creating a 'psql.source' file for the regression
tests instead of just throwing things into 'misc.source'. Seems like we
should probably have more psql-related testing anyway and dumping
everything into 'misc.source' really isn't a good idea.
Thanks!
Stephen
Hi
2017-03-15 17:21 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
Pavel,
I started looking through this to see if it might be ready to commit and
I don't believe it is. Below are my comments about the first patch, I
didn't get to the point of looking at the others yet since this one had
issues.* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-01-09 17:24 GMT+01:00 Jason O'Donnell <odonnelljp01@gmail.com>:
gstore/gbstore:
I don't see the point to 'gstore'- how is that usefully different from
just using '\g'? Also, the comments around these are inconsistent, some
say they can only be used with a filename, others say it could be a
filename or a pipe+command.
\gstore ensure dump row data. It can be replaced by \g with some other
setting, but if query is not unique, then the result can be messy. What is
not possible with \gbstore.
More interesting is \gbstore that uses binary API - it can be used for
bytea fields or for XML fields with implicit correct encoding change.
\gbstore is not possible to replace by \g.
There's a whitespace-only hunk that shouldn't be included.
I don't agree with the single-column/single-row restriction on these. I
can certainly see a case where someone might want to, say, dump out a
bunch of binary integers into a file for later processing.The tab-completion for 'gstore' wasn't correct (you didn't include the
double-backslash). The patch also has conflicts against current master
now.I guess my thinking about moving this forward would be to simplify it to
just '\gb' which will pull the data from the server side in binary
format and dump it out to the filename or command given. If there's a
new patch with those changes, I'll try to find time to look at it.
ok I'll prepare patch
Show quoted text
I would recommend going through a detailed review of the other patches
also before rebasing and re-submitting them also, in particular look to
make sure that the comments are correct and consistent, that there are
comments where there should be (generally speaking, whole functions
should have at least some comments in them, not just the function header
comment, etc).Lastly, I'd suggest creating a 'psql.source' file for the regression
tests instead of just throwing things into 'misc.source'. Seems like we
should probably have more psql-related testing anyway and dumping
everything into 'misc.source' really isn't a good idea.Thanks!
Stephen
Pavel,
* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-03-15 17:21 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
I started looking through this to see if it might be ready to commit and
I don't believe it is. Below are my comments about the first patch, I
didn't get to the point of looking at the others yet since this one had
issues.* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-01-09 17:24 GMT+01:00 Jason O'Donnell <odonnelljp01@gmail.com>:
gstore/gbstore:
I don't see the point to 'gstore'- how is that usefully different from
just using '\g'? Also, the comments around these are inconsistent, some
say they can only be used with a filename, others say it could be a
filename or a pipe+command.\gstore ensure dump row data. It can be replaced by \g with some other
setting, but if query is not unique, then the result can be messy. What is
not possible with \gbstore.
I don't understand what you mean by "the result can be messy." We have
lots of options for controlling the output of the query and those can be
used with \g just fine. This seems like what you're doing is inventing
something entirely new which is exactly the same as setting the right
options which already exist and that seems odd to me.
Is it any different from setting \a and \t and then calling \g? If not,
then I don't see why it would be useful to add.
More interesting is \gbstore that uses binary API - it can be used for
bytea fields or for XML fields with implicit correct encoding change.
\gbstore is not possible to replace by \g.
Yes, having a way to get binary data out using psql and into a file is
interesting and I agree that we should have that capability.
Further, what I think we will definitely need is a way to get binary
data out using psql at the command-line too. We have the -A and -t
switches which correspond to \a and \t, we should have something for
this too. Perhaps what that really calls for is a '\b' and a '-B'
option to go with it which will flip psql into binary mode, similar to
the other Formatting options. I realize it might seem a bit
counter-intuitive, but I can actually see use-cases for having binary
data spit out to $PAGER (when you have a $PAGER that handles it
cleanly, as less does, for example).
There's a whitespace-only hunk that shouldn't be included.
I don't agree with the single-column/single-row restriction on these. I
can certainly see a case where someone might want to, say, dump out a
bunch of binary integers into a file for later processing.The tab-completion for 'gstore' wasn't correct (you didn't include the
double-backslash). The patch also has conflicts against current master
now.I guess my thinking about moving this forward would be to simplify it to
just '\gb' which will pull the data from the server side in binary
format and dump it out to the filename or command given. If there's a
new patch with those changes, I'll try to find time to look at it.ok I'll prepare patch
Great, thanks!
Stephen
2017-03-16 22:01 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
Pavel,
* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-03-15 17:21 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
I started looking through this to see if it might be ready to commit
and
I don't believe it is. Below are my comments about the first patch, I
didn't get to the point of looking at the others yet since this one had
issues.* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-01-09 17:24 GMT+01:00 Jason O'Donnell <odonnelljp01@gmail.com>:
gstore/gbstore:
I don't see the point to 'gstore'- how is that usefully different from
just using '\g'? Also, the comments around these are inconsistent,some
say they can only be used with a filename, others say it could be a
filename or a pipe+command.\gstore ensure dump row data. It can be replaced by \g with some other
setting, but if query is not unique, then the result can be messy. Whatis
not possible with \gbstore.
I don't understand what you mean by "the result can be messy." We have
lots of options for controlling the output of the query and those can be
used with \g just fine. This seems like what you're doing is inventing
something entirely new which is exactly the same as setting the right
options which already exist and that seems odd to me.Is it any different from setting \a and \t and then calling \g? If not,
then I don't see why it would be useful to add.
I am searching some comfortable way - I agree, it can be redundant to
already available functionality.
More interesting is \gbstore that uses binary API - it can be used for
bytea fields or for XML fields with implicit correct encoding change.
\gbstore is not possible to replace by \g.Yes, having a way to get binary data out using psql and into a file is
interesting and I agree that we should have that capability.Further, what I think we will definitely need is a way to get binary
data out using psql at the command-line too. We have the -A and -t
switches which correspond to \a and \t, we should have something for
this too. Perhaps what that really calls for is a '\b' and a '-B'
option to go with it which will flip psql into binary mode, similar to
the other Formatting options. I realize it might seem a bit
counter-intuitive, but I can actually see use-cases for having binary
data spit out to $PAGER (when you have a $PAGER that handles it
cleanly, as less does, for example).
It is interesting idea. I am not sure if it is more formatting option or
general psql option. But can be interesting for another purposes too.
One idea for import files to postgres via psql
we can introduce \gloadfrom that can replace parameters by files - and this
statement can work in text or in binary mode controlled by proposed option.
some like
insert into foo values('Pavel','Stehule', $1) \gloadfrom ~/avatar.jpg
insert into doc(id, doc) values(default, $1) \gloadfrom ~/mydoc.xml
Regards
Pavel
Show quoted text
There's a whitespace-only hunk that shouldn't be included.
I don't agree with the single-column/single-row restriction on these.
I
can certainly see a case where someone might want to, say, dump out a
bunch of binary integers into a file for later processing.The tab-completion for 'gstore' wasn't correct (you didn't include the
double-backslash). The patch also has conflicts against current master
now.I guess my thinking about moving this forward would be to simplify it
to
just '\gb' which will pull the data from the server side in binary
format and dump it out to the filename or command given. If there's a
new patch with those changes, I'll try to find time to look at it.ok I'll prepare patch
Great, thanks!
Stephen
Hi
There's a whitespace-only hunk that shouldn't be included.
I don't agree with the single-column/single-row restriction on these.
I
can certainly see a case where someone might want to, say, dump out a
bunch of binary integers into a file for later processing.The tab-completion for 'gstore' wasn't correct (you didn't include the
double-backslash). The patch also has conflicts against current master
now.I guess my thinking about moving this forward would be to simplify it
to
just '\gb' which will pull the data from the server side in binary
format and dump it out to the filename or command given. If there's a
new patch with those changes, I'll try to find time to look at it.ok I'll prepare patch
Great, thanks!
I rewrote these patches - it allows binary export/import from psql and the
code is very simple. The size of the patch is bigger due including 4KB
binary file (in hex format 8KB).
What is done:
create table foo foo(a bytea);
-- import
insert into foo values($1)
\gloadfrom ~/xxx.jpg bytea
-- export
\pset format binary
select a from foo
\g ~/xxx2.jpg
tested on import 55MB binary file
Comments, notes?
Available import formats are limited to text, bytea, xml - these formats
are safe for receiving data via recv function.
Regards
Pavel
Show quoted text
Stephen
Attachments:
psql-binary-export-import.patchtext/x-patch; charset=US-ASCII; name=psql-binary-export-import.patchDownload
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 2a9c412020..f26d406f89 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -1945,6 +1945,38 @@ CREATE INDEX
<varlistentry>
+ <term><literal>\gloadfrom <replaceable class="parameter">filename</replaceable> <optional> <replaceable class="parameter">format</replaceable> </optional> </literal></term>
+
+ <listitem>
+ <para>
+ Sends the current query input buffer to the server. The current
+ query should to have one query parameter <literal>$1</literal>.
+ The content of <literal>filename</literal> file will be used as
+ the value of this parameter.
+ </para>
+
+ <para>
+ When <literal>format</literal> is not specified, then data are
+ passed in text format. When format is specified, then data are
+ passed in binary format. The available formats are:
+ <literal>text</literal>, <literal>bytea</literal>
+ or <literal>xml</literal> type. In the example the XML document is
+ imported to table <structname>my_table</>:
+<programlisting>
+=> <userinput>CREATE TABLE my_table(id serial, doc xml);</>
+=> <userinput>INSERT INTO my_table(doc) VALUES($1) RETURNING id</>
+-> <userinput>\gloadfrom ~/Documents/data.xml xml</>
+ id
+----
+ 1
+(1 row)
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
<term><literal>\gset [ <replaceable class="parameter">prefix</replaceable> ]</literal></term>
<listitem>
@@ -2366,8 +2398,8 @@ lo_import 152801
<literal>aligned</literal>, <literal>wrapped</literal>,
<literal>html</literal>, <literal>asciidoc</literal>,
<literal>latex</literal> (uses <literal>tabular</literal>),
- <literal>latex-longtable</literal>, or
- <literal>troff-ms</literal>.
+ <literal>latex-longtable</literal>, <literal>troff-ms</literal> or
+ <literal>binary</literal>.
Unique abbreviations are allowed. (That would mean one letter
is enough.)
</para>
@@ -2404,6 +2436,12 @@ lo_import 152801
also requires the <application>LaTeX</application>
<literal>longtable</literal> and <literal>booktabs</> packages.
</para>
+
+ <para>
+ The <literal>binary</> format is simply output values in
+ binary format.
+ </para>
+
</listitem>
</varlistentry>
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 4f4a0aa9bd..51ed3df58c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -28,6 +28,7 @@
#endif
#include "catalog/pg_class.h"
+#include "catalog/pg_type.h"
#include "portability/instr_time.h"
#include "libpq-fe.h"
@@ -936,6 +937,50 @@ exec_command(const char *cmd,
status = PSQL_CMD_SEND;
}
+ /*
+ * \gloadfrom filename -- send query and use content of file as parameter
+ */
+ else if (strcmp(cmd, "gloadfrom") == 0)
+ {
+ char *fmt = NULL;
+ char *fname = psql_scan_slash_option(scan_state,
+ OT_FILEPIPE, NULL, false);
+
+ if (!fname)
+ {
+ psql_error("\\%s: missing required argument\n", cmd);
+ success = false;
+ }
+ else
+ {
+ /* try to get separate format arg */
+ fmt = psql_scan_slash_option(scan_state,
+ OT_NORMAL, NULL, true);
+
+ if (fmt)
+ {
+ if (strcmp(fmt, "text") == 0)
+ pset.gloadfrom_fmt = TEXTOID;
+ else if (strcmp(fmt, "bytea") == 0)
+ pset.gloadfrom_fmt = BYTEAOID;
+ else if (strcmp(fmt, "xml") == 0)
+ pset.gloadfrom_fmt = XMLOID;
+ else
+ {
+ psql_error("\\%s: only [text, bytea, xml] format can be specified\n", cmd);
+ success = false;
+ }
+ }
+ else
+ pset.gloadfrom_fmt = 0; /* UNKNOWNOID */
+
+ expand_tilde(&fname);
+ pset.gloadfrom = pg_strdup(fname);
+ }
+ free(fname);
+ status = PSQL_CMD_SEND;
+ }
+
/* \gset [prefix] -- send query and store result into variables */
else if (strcmp(cmd, "gset") == 0)
{
@@ -2518,6 +2563,9 @@ _align2string(enum printFormat in)
case PRINT_TROFF_MS:
return "troff-ms";
break;
+ case PRINT_BINARY:
+ return "binary";
+ break;
}
return "unknown";
}
@@ -2589,9 +2637,11 @@ do_pset(const char *param, const char *value, printQueryOpt *popt, bool quiet)
popt->topt.format = PRINT_LATEX_LONGTABLE;
else if (pg_strncasecmp("troff-ms", value, vallen) == 0)
popt->topt.format = PRINT_TROFF_MS;
+ else if (pg_strncasecmp("binary", value, vallen) == 0)
+ popt->topt.format = PRINT_BINARY;
else
{
- psql_error("\\pset: allowed formats are unaligned, aligned, wrapped, html, asciidoc, latex, latex-longtable, troff-ms\n");
+ psql_error("\\pset: allowed formats are unaligned, aligned, wrapped, html, asciidoc, latex, latex-longtable, troff-ms, binary\n");
return false;
}
}
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index e9d4fe6786..d9a52d17e6 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -12,6 +12,7 @@
#include <limits.h>
#include <math.h>
#include <signal.h>
+#include <sys/stat.h>
#ifndef WIN32
#include <unistd.h> /* for write() */
#else
@@ -32,6 +33,7 @@
static bool ExecQueryUsingCursor(const char *query, double *elapsed_msec);
static bool command_no_begin(const char *query);
static bool is_select_command(const char *query);
+static bool read_file(char *fname, PQExpBuffer rawbuf);
/*
@@ -1285,7 +1287,8 @@ SendQuery(const char *query)
}
if (pset.fetch_count <= 0 || pset.gexec_flag ||
- pset.crosstab_flag || !is_select_command(query))
+ pset.crosstab_flag || !is_select_command(query) ||
+ pset.popt.topt.format == PRINT_BINARY || pset.gloadfrom)
{
/* Default fetch-it-all-and-print mode */
instr_time before,
@@ -1294,7 +1297,56 @@ SendQuery(const char *query)
if (pset.timing)
INSTR_TIME_SET_CURRENT(before);
- results = PQexec(pset.db, query);
+ /*
+ * PQexecParams disallow multiple commands often
+ * used by -c option. From compatibility reason the
+ * PQexecParams is used only when it is necessary.
+ */
+ if (pset.popt.topt.format == PRINT_BINARY || pset.gloadfrom)
+ {
+ if (pset.gloadfrom)
+ {
+ PQExpBufferData rawbuf;
+
+ initPQExpBuffer(&rawbuf);
+
+ if (read_file(pset.gloadfrom, &rawbuf))
+ {
+ Oid paramTypes[1];
+ const char *paramValues[1];
+ int paramLengths[1];
+ int paramFormats[1];
+
+ paramTypes[0] = pset.gloadfrom_fmt;
+ paramFormats[0] = pset.gloadfrom_fmt != 0 ? 1 : 0;
+ paramValues[0] = rawbuf.data;
+ paramLengths[0] = rawbuf.len;
+
+ results = PQexecParams(pset.db, query,
+ 1,
+ paramTypes,
+ paramValues,
+ paramLengths,
+ paramFormats,
+ pset.popt.topt.format == PRINT_BINARY);
+ }
+ else
+ {
+ OK = false;
+ results = NULL;
+ }
+
+ termPQExpBuffer(&rawbuf);
+ }
+ else
+ {
+ results = PQexecParams(pset.db, query,
+ 0, NULL, NULL, NULL, NULL,
+ pset.popt.topt.format == PRINT_BINARY);
+ }
+ }
+ else
+ results = PQexec(pset.db, query);
/* these operations are included in the timing result: */
ResetCancelConn();
@@ -1418,6 +1470,13 @@ sendquery_cleanup:
pset.gfname = NULL;
}
+ /* reset \gloadfrom input-from-filename trgger */
+ if (pset.gloadfrom)
+ {
+ free(pset.gloadfrom);
+ pset.gloadfrom = NULL;
+ }
+
/* reset \gx's expanded-mode flag */
pset.g_expanded = false;
@@ -2152,3 +2211,53 @@ recognized_connection_string(const char *connstr)
{
return uri_prefix_length(connstr) != 0 || strchr(connstr, '=') != NULL;
}
+
+/*
+ * read file into buffer
+ */
+static bool
+read_file(char *fname, PQExpBuffer rawbuf)
+{
+ FILE *fd;
+ bool result = false;
+
+ fd = fopen(fname, PG_BINARY_R);
+ if (fd)
+ {
+ struct stat fst;
+
+ if (fstat(fileno(fd), &fst) != -1)
+ {
+ if (S_ISREG(fst.st_mode))
+ {
+ if (fst.st_size <= ((int64) 1024) * 1024 * 1024)
+ {
+ size_t size;
+ char buf[512];
+
+ while ((size = fread(buf, 1, sizeof(buf), fd)) > 0)
+ appendBinaryPQExpBuffer(rawbuf, buf, size);
+
+ if (ferror(fd))
+ psql_error("%s: %s\n", fname, strerror(errno));
+ else if (PQExpBufferBroken(rawbuf))
+ psql_error("out of memory\n");
+ else
+ result = true;
+ }
+ else
+ psql_error("%s is too big (greather than 1GB)\n", fname);
+ }
+ else
+ psql_error("%s is not regular file\n", fname);
+ }
+ else
+ psql_error("%s: %s\n", fname, strerror(errno));
+
+ fclose(fd);
+ }
+ else
+ psql_error("%s: %s\n", fname, strerror(errno));
+
+ return result;
+}
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index ba14df0344..17d0469697 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -167,7 +167,7 @@ slashUsage(unsigned short int pager)
* Use "psql --help=commands | wc" to count correctly. It's okay to count
* the USE_READLINE line even in builds without that.
*/
- output = PageOutput(113, pager ? &(pset.popt.topt) : NULL);
+ output = PageOutput(114, pager ? &(pset.popt.topt) : NULL);
fprintf(output, _("General\n"));
fprintf(output, _(" \\copyright show PostgreSQL usage and distribution terms\n"));
@@ -175,6 +175,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\g [FILE] or ; execute query (and send results to file or |pipe)\n"));
fprintf(output, _(" \\gx [FILE] as \\g, but forces expanded output mode\n"));
fprintf(output, _(" \\gexec execute query, then execute each value in its result\n"));
+ fprintf(output, _(" \\gloadfrom FILE [fmt] execute query, use content of FILE as query parameter\n"));
fprintf(output, _(" \\gset [PREFIX] execute query and store results in psql variables\n"));
fprintf(output, _(" \\q quit psql\n"));
fprintf(output, _(" \\crosstabview [COLUMNS] execute query and display results in crosstab\n"));
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 70ff1812c8..8078890a10 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -91,6 +91,9 @@ typedef struct _psqlSettings
printQueryOpt popt;
char *gfname; /* one-shot file output argument for \g */
+ char *gloadfrom; /* one-shot file input argument for \gloadfrom */
+ Oid gloadfrom_fmt; /* one-shot data file format UNKNOWNOID,
+ * TEXTOID, BYTEAOID or XMLOID */
bool g_expanded; /* one-shot expanded output requested via \gx */
char *gset_prefix; /* one-shot prefix argument for \gset */
bool gexec_flag; /* one-shot flag to execute query's results */
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f7494065de..3eb24c4502 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1410,8 +1410,9 @@ psql_completion(const char *text, int start, int end)
"\\dm", "\\dn", "\\do", "\\dO", "\\dp", "\\drds", "\\ds", "\\dS",
"\\dt", "\\dT", "\\dv", "\\du", "\\dx", "\\dy",
"\\e", "\\echo", "\\ef", "\\encoding", "\\errverbose", "\\ev",
- "\\f", "\\g", "\\gexec", "\\gset", "\\gx", "\\h", "\\help", "\\H",
- "\\i", "\\ir", "\\l", "\\lo_import", "\\lo_export", "\\lo_list",
+ "\\f", "\\g", "\\gexec", "\\gloadfrom", "\\gset", "\\gx",
+ "\\h", "\\help", "\\H", "\\i", "\\ir", "\\l",
+ "\\lo_import", "\\lo_export", "\\lo_list",
"\\lo_unlink", "\\o", "\\p", "\\password", "\\prompt", "\\pset", "\\q",
"\\qecho", "\\r", "\\s", "\\set", "\\setenv", "\\sf", "\\sv", "\\t",
"\\T", "\\timing", "\\unset", "\\x", "\\w", "\\watch", "\\z", "\\!",
@@ -3422,7 +3423,7 @@ psql_completion(const char *text, int start, int end)
{
static const char *const my_list[] =
{"unaligned", "aligned", "wrapped", "html", "asciidoc",
- "latex", "latex-longtable", "troff-ms", NULL};
+ "latex", "latex-longtable", "troff-ms", "binary", NULL};
COMPLETE_WITH_LIST_CS(my_list);
}
@@ -3443,8 +3444,8 @@ psql_completion(const char *text, int start, int end)
}
else if (TailMatchesCS2("\\set", MatchAny))
{
- if (TailMatchesCS1("AUTOCOMMIT|ON_ERROR_STOP|QUIET|"
- "SINGLELINE|SINGLESTEP"))
+ if (TailMatchesCS1("AUTOCOMMIT|BINARY_PARAM"
+ "ON_ERROR_STOP|QUIET|SINGLELINE|SINGLESTEP"))
COMPLETE_WITH_LIST_CS2("on", "off");
else if (TailMatchesCS1("COMP_KEYWORD_CASE"))
COMPLETE_WITH_LIST_CS4("lower", "upper",
@@ -3469,7 +3470,7 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_views, NULL);
else if (TailMatchesCS1("\\cd|\\e|\\edit|\\g|\\i|\\include|"
"\\ir|\\include_relative|\\o|\\out|"
- "\\s|\\w|\\write|\\lo_import"))
+ "\\s|\\w|\\write|\\lo_import|\\gloadfrom"))
{
completion_charp = "\\";
matches = completion_matches(text, complete_from_files);
diff --git a/src/fe_utils/print.c b/src/fe_utils/print.c
index 9180b90004..829f9ebc83 100644
--- a/src/fe_utils/print.c
+++ b/src/fe_utils/print.c
@@ -3293,6 +3293,40 @@ printQuery(const PGresult *result, const printQueryOpt *opt,
if (cancel_pressed)
return;
+ /* direct print in binary format */
+ if (opt->topt.format == PRINT_BINARY)
+ {
+ /* check format */
+ for (c = 0; c < PQnfields(result); c++)
+ if (PQfformat(result, c) == 0)
+ {
+ fprintf(stderr, _("invalid data format (internal error)"));
+ exit(EXIT_FAILURE);
+ }
+
+ for (r = 0; r < PQntuples(result); r++)
+ {
+ for (c = 0; c < PQnfields(result); c ++)
+ {
+ if (!PQgetisnull(result, r, c))
+ {
+ int size = PQgetlength(result, r, c);
+ char *value = PQgetvalue(result, r, c);
+ bool success;
+
+ success = fwrite(value, 1, size, fout) == size;
+ if (!success)
+ {
+ fprintf(stderr, _("write error (internal error): %s"),
+ strerror(errno));
+ exit(EXIT_FAILURE);
+ }
+ }
+ }
+ }
+ return;
+ }
+
printTableInit(&cont, &opt->topt, opt->title,
PQnfields(result), PQntuples(result));
diff --git a/src/include/fe_utils/print.h b/src/include/fe_utils/print.h
index d89b6febcb..64ea406d5c 100644
--- a/src/include/fe_utils/print.h
+++ b/src/include/fe_utils/print.h
@@ -33,7 +33,8 @@ enum printFormat
PRINT_ASCIIDOC,
PRINT_LATEX,
PRINT_LATEX_LONGTABLE,
- PRINT_TROFF_MS
+ PRINT_TROFF_MS,
+ PRINT_BINARY
/* add your favourite output format here ... */
};
diff --git a/src/test/regress/data/logo.data b/src/test/regress/data/logo.data
new file mode 100644
index 0000000000..b324eb2178
--- /dev/null
+++ b/src/test/regress/data/logo.data
@@ -0,0 +1 @@
+\x47494638396164003a00e600002b89bb007bb3005a9b93cfe3006595008ab2458ab2eef5f9008abb0094c20054940079a8a4d7e888bad699c2d83492be297aaa45a5c80065aa77a8c577bad40085b1dbebf30083b90082ac006ba4005da00076a9007ca9bbdbeac6e6f10068adcce3ef0074a40079ad1475acbbd3e40075ac429bc40081b00085ae66a9cb0074b10090be1b7bb3006da952a4c90061a20064a57cc3dd0071ad0061a5f8fcfd0071a8bce1ee006baf0085b6006daf0081b6b1dceb007dae007db700548d126da80068a91482b665b8d65ab3d5005698007fab0058960078a8ccddea004e8c0c65a2669abc0066a00c6dad66a0c24480ac77b2d1004f92bbd7e8aad0e40c68a7aacbde0560a2246ea152accf77adcf66afcf11689fbbcfe20664a20767a9e0f1f6115f96085e9f3d3d3dffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000021f90400000000002c0000000064003a000007ff806282838485868788898a8b8c8d8a092b91910894083897973a22359c9c2d192f46a2a3a4a20a46a70aaaabacadab464c404c19b435013cb701273d273abe989795082bc3922b09c80982c8c6c294983d25d29d194c0246d7a5a5a7a8ddaca8aeab02b1b4b425b93cbc3abd17c0cec691c9c8cb9092ce983ad22535329cd6d8da027a1b984a15b8561a320029d742448087bb7ae9c03131983062c6981540b10cdebd4b224ac810c9cf9a806b27491119888a88ab243093847389eac5c2852d5ae0d2d5c397af0bed705c2c26a90086a318e879144631a43e192d3408207032a5001faa562af011856b14054a942888f2e38189080f9424f9da8da628269fff7236e429b122a677933014298294a31848f52609bb20c2e9481930a8564599644b0c1b1d76d8b011830504063b32eff87280f2801d1d06b0f84aa4b469231a5a0091db22408f9e3e81debd88a30887bd7c912ab547ac922615fa5464902a75b10f16166240d0a0800a002c340e985012c5c8081703188c8019c6850717594b1b310d236e4e113d78f8fc69b1120e1e1ce2e3469af42fb3669502885011123893171a04589c023f58f0804ca624f1c0003f24b1122a6b3d60a04c4928614304513c281e6aace5f45a6cedb4e31e070bc427df7c47d113586f945ca01f7fc0b500e054020c880505a7540596052c240110404424018105006498040b1e88f5e0ff2802ac26970cae8178010253e240e202259a885b6ef525b314020f05006309a01010608d553150e4620a4410838f5589720d115100604318aa08b0c308dc9022c00b1dcab09e0eb2bd77c4a15866791b8a5d7ac91b9861ee27020c2f0058e9995dd8008002689e64c381a62d26804b49c430c45761ec1985a827a5d6025432c8d01350b29db0c106871e8125892772d9e87df6e010a99825546aa9800258c88202677a0a010b0f0080928e2c4430800761443182073f284004ab0264006bac01104aab081b84706bae89f6ba578af601db9b0e618a29828cc61e2b40173b6c4adcbe5fece0c1183460f16d8d0a3c70c018072459e714a3822b000c32e410abff0c2a98dbc3adb7aa9babae596eb9d7092a3a4acc6bf506d0c27ff95e2a400c0f307b2615dcb270c0171e5851a306510c70adb5746281c5aa12537c31c684f2c071baebb21bf296f0cab3220228d70b04a52d57daa610a80968c500a7281106833b2b3080090ab030c0aa9a725ad5bf1abc7074ace995b081dd4b7facab891cf0c065c9c0525df5435763dd3211765a81ac150ca02640cf00744d0416032401c0da2cec1046ab68c23df748772fdd31ae4e6bc9175f80df27b80e3d84290b0cb01b0e83004dd8f043dc008ec0c09fcc0df0c0ceb473ebc5daa6ca2ce0f101aa8071ac19d41092e8a47fbc6baf514b5d0f45af551d7bec2f184e84ef7f020840ffe3026ab7b306460c11c3d73f78d0c4f9c80ba8bcf2389580eed21e976e7a6ea9d77301ebac734d006050b8ed756f0602a01c116690bb1d280e71390b10808850bb0708c15444885ffc9427035a2c6403f7c31fe9da653addc4cb7aff635df67a200b2014707b1ad896173430831a32c0055f8981c12c35030d1021023863001532a84109be4079aa294761a0d73490edcf84154080f556001415664f35af7b210c10784322504a002338801000f0851110a16501dac1181e10052bcc306e1234e20c54909372302184d04394a216451f4154e08f529447152fb04219b8d085db8b9d061e10c11910e1013b18c20e68308011d0b0523dfc010392a33edb05a8ff87a08c9b04a052478580308f47c0150917f5aefafcf18f2718063268454815aae090b82420ec5e300306188c0826f00000326782cf0880810c24c2102af98021346c8659ab14c5e4a2443c3611517c839a2b61f9ca61506990afb9252e5d288172c240021a408e593c10030f30c00454b081072656431828f30bd91940043e59c37ed6b07bb12a650b42c7b1544a6f7a7c34210a5e5981135420133da0554f5e334e5c4a40971a00c00e1850ade40c60320398d8167b180313b0600408a4a13ffde92a6a02a10604159d1ef9c64a2832f48f14b1956b247a811ce4a4a2872ca70684aa81200000004d28a6002e0a030d7c60072cc8603d61d74feecd20a0a5ffb41b1e53394284be2b377eb969432bb00efb3924a284a4630b3e005472321508e83c93091830d473a6d3062ca8ebf66640d52d6a000858fd44064088b76b824c51ad3c4a586f8a8313e0a0026675886b5e93831bdc60ad1fc86c6681c0d60f94f3b3e59c0124872a548d36a1861298416a57ab5a09f0f255a414ac59f316bd55a20805db64e8634152987a85330795b5ac70354bdce2ce40a3a4fd404607f081d68216b40c1c65603fb144dace747fbef223631ffa9edefa5605976d81702d5bdcf296330736582d3a853004d23ef7b3fd6c0107a9e93c8e89c0a0072d2197162bd64c3844b293ed810c2e3bde0293b7b8e58c240d3560020b044102e685efff0c80a082f9d6d17ea864573619c55fc63e563f616a1dca7c6a60039b37083660808a853983f26696a97c5dde7441684dbded519b28e8f043717a891300f821d9036f894bfc811b6876064d88001600504e177b16c637a87040a19281d9a212a127a24f8efd5880ed62e2c701ee0178810bdcf10677c8cf2d7279e17ad119c857cab16d4861ee5b5044ed0dbb8addb2182ad065dd72b7b13f5ea1986f50d9320f99c8e38df045dfac0200a4000a5048c108acbc81074081025078c002eefcc43c87b5cf111883a847dd01218059d02a30f4998f7a68231bd9c59f25608555d080518bfa002998b3846c3d0603ed6a0c14c0718efdd2e52e87da0113c8c2041aff6081314021c482665daa094de61c48410a1268f570115cce2857180a6320010460f204241c00022250d8019600061f806102073880a63930860964402f7d19b6208a5d81501b20044708c102c278800cf8367b841473b52b2b051210e1cc4486b50472306b1574600c6050c06a3260827a87e0e210f00113041e0200843b04f4b6f77e874dec02187b0c0628c2427150001b8ce10522d00208440d022d648f02031b0308a000800ec49b040dc80100a6c0731734a0013778fa08b22005008cc0010b3bc0141e50e18b83e1a555168109ae00813138c00700dfb4ae1c30060814c1e32bd7b7185cce677f7360c738e00c01626076033ca1d60de80105c66002ff18286102637042162c6081092ce10af19ec0139660810390400052400212a430812b983b0b063080b959a082148c0109296081f3ee5b03d32f8100a5d382c7eb7d6fc56280e5fb767901423d8108f85e08176f40d9a1008b5865610c2ca0000834a082d764e10933200117a2a001b63f21431ab8c218b8a00029d43b0a517002cc8920811684d101afcac2c21836050a04e108e07e4206049ea80574dcde70bf3dcbf5acfbddf39a610d4000a6070106f7102637012e607611f01a3d95031de0701f200507100533e06a52307d12980473340063f00426f0812600021620002d500361700513e000cd76002e3078f2775825727f19e0715a867b73a77bbc17ff7aa14700041002834700fa2159f576032e40023c4701afd101523047e116059a75030d27000f48040f7171ff37063e50828355034c40004bd06ba6370169373d83b7043328861ca07f2c5700b9e772feb600b871773830783fd014221004f506040120032f60008807053da0843010880730032bf4802fa0841a202617a70a58111357200529603f0b705f47b073da47020490285a30001c707157100234e8691bc18637e88630277337d5714ef050271000b2070150a0056122030dd7024af80201307806206241700052a0010fc87cb4d6811cd47a92d66b1950184720020b50793ec0764e50220ba03046e80046907221e06928a07b6de87f0670770ce5ff5046e7028f85055ae70353700000f0104160010e908848a08bbf78005060025af005164002c2c88f15667248300235d0021d97054c506b24f000ce08006cd7003e10044860762e60022e707107b00504f07624706914d09114d08d2ef78dfe860162550141c076a3e6005bf0024160845270711db00507296a1d100027b93020e0025cc08fc35802ca63025c300652b0730d90044cc001ea676b0790053eb08937696b55506f69788563b000de788a2e870119599215800223107a1899013c10125bf004067005a8710e00f0046070442a100a9c620155200025701222d10f4c70054fc09622775f22400010b0974fe0034d09322140007e670046e0034e10ff907ac18390099957d97fbac7950e75023ec601bdd00b3c200d25280d4e5182bfd80100a00227d9811a201224d10982055374b64421307ff47758f11102b67914234000f4a186a5189273b70294d967627599c4e95827c0039da90fca290dbb766b4ee003e2c20fac395025c00185616b1b606b473006dbb90063406fa3f676a38601b68602a3d67ffb069cc1699998599ced890bcba99c1910988329729de00935603fd63967dc2902dcc99ddd5995df39a0273206473106e6c97263e0720bea9bea49997c5692eed99e98b90ec7e914f9b90ffd500dcd739ff95918f0316795a89d002aa0de496fe039067b319ee6296a39d6a00d6a8a05009c0f5a99c209c44b98890114baa3ef890bcfb30f9d803773b69f855189ceb89dec52a2274aa02afa2e069aa008ba500500a39339a3c1c96f7fb4509859013acaa3ef799c608a9c622aa6206a9d446aa4b902a0ba626bde696be2296ae4797b086a6b532a6ae82906347aa574f74a0b459214daa5ef5904c739a6848a9c7de38c22b09f95b8696ac7a8f58765dac48d39c6677a6a8a795aa3fdc7503976023a0aa82720a83c00a64550a8863a96448aa88b8a2530d82e7bc44a89c58dbda9a78e30abb45aabb67aab821008003b
diff --git a/src/test/regress/input/psql_export_import.source b/src/test/regress/input/psql_export_import.source
new file mode 100644
index 0000000000..a6545c6de5
--- /dev/null
+++ b/src/test/regress/input/psql_export_import.source
@@ -0,0 +1,29 @@
+-- export/import binary varlena objects
+create table test_bin_objects(id integer, doc bytea);
+insert into test_bin_objects values(1, E'\\xDE00ADBEEF');
+
+\pset format binary
+select doc from test_bin_objects where id = 1
+\g results/varlenatest.bin
+
+insert into test_bin_objects values(2, $1)
+\gloadfrom results/varlenatest.bin bytea
+
+\pset format aligned
+select id, doc, md5(doc) from test_bin_objects;
+
+truncate test_bin_objects;
+
+insert into test_bin_objects values(1, $1)
+\gloadfrom '@abs_srcdir@/data/logo.data'
+
+\pset format binary
+select doc from test_bin_objects where id = 1
+\g results/100x58_1.gif
+
+insert into test_bin_objects values(2, $1)
+\gloadfrom results/100x58_1.gif bytea
+
+\pset format aligned
+-- print when md5 is unexpected (result should be empty)
+select id, md5(doc) from test_bin_objects where md5(doc) <> '24da0c4de68bf19682344f8b060faeb5';
diff --git a/src/test/regress/output/psql_export_import.source b/src/test/regress/output/psql_export_import.source
new file mode 100644
index 0000000000..bd7dbb3692
--- /dev/null
+++ b/src/test/regress/output/psql_export_import.source
@@ -0,0 +1,31 @@
+-- export/import binary varlena objects
+create table test_bin_objects(id integer, doc bytea);
+insert into test_bin_objects values(1, E'\\xDE00ADBEEF');
+\pset format binary
+select doc from test_bin_objects where id = 1
+\g results/varlenatest.bin
+insert into test_bin_objects values(2, $1)
+\gloadfrom results/varlenatest.bin bytea
+\pset format aligned
+select id, doc, md5(doc) from test_bin_objects;
+ id | doc | md5
+----+--------------+----------------------------------
+ 1 | \xde00adbeef | 70afdd78c75bfb6e76781d3785858b66
+ 2 | \xde00adbeef | 70afdd78c75bfb6e76781d3785858b66
+(2 rows)
+
+truncate test_bin_objects;
+insert into test_bin_objects values(1, $1)
+\gloadfrom '@abs_srcdir@/data/logo.data'
+\pset format binary
+select doc from test_bin_objects where id = 1
+\g results/100x58_1.gif
+insert into test_bin_objects values(2, $1)
+\gloadfrom results/100x58_1.gif bytea
+\pset format aligned
+-- print when md5 is unexpected (result should be empty)
+select id, md5(doc) from test_bin_objects where md5(doc) <> '24da0c4de68bf19682344f8b060faeb5';
+ id | md5
+----+-----
+(0 rows)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ea7b5b4aa2..053ee03ed3 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -89,7 +89,7 @@ test: brin gin gist spgist privileges init_privs security_label collate matview
# ----------
# Another group of parallel tests
# ----------
-test: alter_generic alter_operator misc psql async dbsize misc_functions sysviews tsrf
+test: alter_generic alter_operator misc psql psql_export_import async dbsize misc_functions sysviews tsrf
# rules cannot run concurrently with any test that creates a view
test: rules psql_crosstab amutils
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index cf48ea7cc8..32aaa30e15 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -124,6 +124,7 @@ test: alter_generic
test: alter_operator
test: misc
test: psql
+test: psql_export_import
test: async
test: dbsize
test: misc_functions
Hi,
On 2017-03-18 17:51:48 +0100, Pavel Stehule wrote:
What is done:
create table foo foo(a bytea);
-- import
insert into foo values($1)
\gloadfrom ~/xxx.jpg bytea-- export
\pset format binary
select a from foo
\g ~/xxx2.jpgtested on import 55MB binary file
Comments, notes?
Available import formats are limited to text, bytea, xml - these formats
are safe for receiving data via recv function.
I don't think we have design agreement on this at this point. Given the
upcoming code freeze, I think we'll have to hash this out during the
next development cycle. Any counterarguments?
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2017-03-18 17:51:48 +0100, Pavel Stehule wrote:
What is done:
create table foo foo(a bytea);
-- import
insert into foo values($1)
\gloadfrom ~/xxx.jpg bytea-- export
\pset format binary
select a from foo
\g ~/xxx2.jpgtested on import 55MB binary file
Comments, notes?
I don't like the API here much. Loading requires knowledge of some
magic $1 value and allows only a single column, printing doesn't mean
much when there's multiple columns / rows.
I think the loading side of things should be redesigned into a more
general facility for providing query parameters. E.g. something like
\setparam $1 'whateva'
\setparamfromfile $2 'somefile'
\setparamfromprogram $3 cat /frakbar
which then would get used in the next query sent to the server. That'd
allow importing multiple columns, and it'd be useful for other purposes
than just loading binary data.
I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded. One approach would be to to have
something like
\storequeryresult filename_template.%row.%column
which'd then print the current query buffer into the relevant file after
doing replacement on %row and %column.
I don't think we can find an API we agree upon in the next 48h...
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Andres,
* Andres Freund (andres@anarazel.de) wrote:
I don't like the API here much. Loading requires knowledge of some
magic $1 value and allows only a single column, printing doesn't mean
much when there's multiple columns / rows.
I think the loading side of things should be redesigned into a more
general facility for providing query parameters. E.g. something like
\setparam $1 'whateva'
\setparamfromfile $2 'somefile'
\setparamfromprogram $3 cat /frakbarwhich then would get used in the next query sent to the server. That'd
allow importing multiple columns, and it'd be useful for other purposes
than just loading binary data.
I tend to agree that the loading side should probably be thought through
more.
I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded. One approach would be to to have
something like\storequeryresult filename_template.%row.%column
which'd then print the current query buffer into the relevant file after
doing replacement on %row and %column.
I don't actually agree that there's a problem having the output from a
query stored direclty in binary form into a single file. The above
approach seems to imply that every binary result must go into an
independent file, and perhaps that would be useful in some cases, but I
don't see it as required.
I don't think we can find an API we agree upon in the next 48h...
Now that there's more than one opinion being voiced on the API, I tend
to agree with this. Hopefully we can keep the discussion moving
forward, however, as I do see value in this capability in general.
Thanks!
Stephen
Hi,
On 2017-04-05 21:07:59 -0400, Stephen Frost wrote:
* Andres Freund (andres@anarazel.de) wrote:
I don't like the API here much. Loading requires knowledge of some
magic $1 value and allows only a single column, printing doesn't mean
much when there's multiple columns / rows.I think the loading side of things should be redesigned into a more
general facility for providing query parameters. E.g. something like
\setparam $1 'whateva'
\setparamfromfile $2 'somefile'
\setparamfromprogram $3 cat /frakbarwhich then would get used in the next query sent to the server. That'd
allow importing multiple columns, and it'd be useful for other purposes
than just loading binary data.I tend to agree that the loading side should probably be thought through
more.I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded. One approach would be to to have
something like\storequeryresult filename_template.%row.%column
which'd then print the current query buffer into the relevant file after
doing replacement on %row and %column.I don't actually agree that there's a problem having the output from a
query stored direclty in binary form into a single file. The above
approach seems to imply that every binary result must go into an
independent file, and perhaps that would be useful in some cases, but I
don't see it as required.
Well, it'd not be enforced - it'd depend on your template. But for a
lot of types of files, it'd not make sense to store multiple
columns/rows in file. Particularly for ones where printing them out to
files is actually meaningful (i.e. binary ones).
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Andres,
* Andres Freund (andres@anarazel.de) wrote:
On 2017-04-05 21:07:59 -0400, Stephen Frost wrote:
* Andres Freund (andres@anarazel.de) wrote:
I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded. One approach would be to to have
something like\storequeryresult filename_template.%row.%column
which'd then print the current query buffer into the relevant file after
doing replacement on %row and %column.I don't actually agree that there's a problem having the output from a
query stored direclty in binary form into a single file. The above
approach seems to imply that every binary result must go into an
independent file, and perhaps that would be useful in some cases, but I
don't see it as required.Well, it'd not be enforced - it'd depend on your template. But for a
lot of types of files, it'd not make sense to store multiple
columns/rows in file. Particularly for ones where printing them out to
files is actually meaningful (i.e. binary ones).
Having the template not require the row/column place-holders included
strikes me as more likely to be confusing. My initial thinking around
this was that users who actually want independent files would simply
issue independent queries, while users who want to take a bunch of int4
columns and dump them into a single binary file would be able to do so
easily.
I'm not against adding the ability for a single query result to be saved
into independent files, but it strikes me as feature creep on this basic
capability. Further, I don't see any particular reason why splitting up
the output from a query into multiple files is only relevant for binary
data.
Thanks!
Stephen
2017-04-06 3:34 GMT+02:00 Stephen Frost <sfrost@snowman.net>:
Andres,
* Andres Freund (andres@anarazel.de) wrote:
On 2017-04-05 21:07:59 -0400, Stephen Frost wrote:
* Andres Freund (andres@anarazel.de) wrote:
I don't yet have a good idea how to deal with moving individual cells
into files, so they can be loaded. One approach would be to to have
something like\storequeryresult filename_template.%row.%column
which'd then print the current query buffer into the relevant file
after
doing replacement on %row and %column.
I don't actually agree that there's a problem having the output from a
query stored direclty in binary form into a single file. The above
approach seems to imply that every binary result must go into an
independent file, and perhaps that would be useful in some cases, but I
don't see it as required.Well, it'd not be enforced - it'd depend on your template. But for a
lot of types of files, it'd not make sense to store multiple
columns/rows in file. Particularly for ones where printing them out to
files is actually meaningful (i.e. binary ones).Having the template not require the row/column place-holders included
strikes me as more likely to be confusing. My initial thinking around
this was that users who actually want independent files would simply
issue independent queries, while users who want to take a bunch of int4
columns and dump them into a single binary file would be able to do so
easily.I'm not against adding the ability for a single query result to be saved
into independent files, but it strikes me as feature creep on this basic
capability. Further, I don't see any particular reason why splitting up
the output from a query into multiple files is only relevant for binary
data.
The files can be simply joined together outside psql
Personally I prefer relation type: single field, single file in special g
command - because I can simply off all formatting and result should be
correct every time.
Stephen, have you some use case for your request?
Regards
Pavel
Show quoted text
Thanks!
Stephen
Greetings,
* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-04-06 3:34 GMT+02:00 Stephen Frost <sfrost@snowman.net>:
Having the template not require the row/column place-holders included
strikes me as more likely to be confusing. My initial thinking around
this was that users who actually want independent files would simply
issue independent queries, while users who want to take a bunch of int4
columns and dump them into a single binary file would be able to do so
easily.I'm not against adding the ability for a single query result to be saved
into independent files, but it strikes me as feature creep on this basic
capability. Further, I don't see any particular reason why splitting up
the output from a query into multiple files is only relevant for binary
data.The files can be simply joined together outside psql
Just as multiple queries could be done to have the results put into
independent files.
Personally I prefer relation type: single field, single file in special g
command - because I can simply off all formatting and result should be
correct every time.
Not sure why you think there would be a formatting issue or why the
result might not be 'correct'.
Stephen, have you some use case for your request?
The initial patch forced a single value result. Including such a
restriction doesn't seem necessary to me. As for use-case, I've
certainly written code to work with binary-result data from PG
previously and it seems entirely reasonable that someone might wish to
pull data into a file with psql and then process it. I've been
wondering if we should consider how binary-mode COPY works, but that
format ends up being pretty inefficient due to the repeated 32-bit
length value for every field.
My initial reaction was primairly that I didn't see value in the
somewhat arbitrary restriction being imposed on usage of this.
Thanks!
Stephen
2017-04-06 14:47 GMT+02:00 Stephen Frost <sfrost@snowman.net>:
Greetings,
* Pavel Stehule (pavel.stehule@gmail.com) wrote:
2017-04-06 3:34 GMT+02:00 Stephen Frost <sfrost@snowman.net>:
Having the template not require the row/column place-holders included
strikes me as more likely to be confusing. My initial thinking around
this was that users who actually want independent files would simply
issue independent queries, while users who want to take a bunch of int4
columns and dump them into a single binary file would be able to do so
easily.I'm not against adding the ability for a single query result to be
saved
into independent files, but it strikes me as feature creep on this
basic
capability. Further, I don't see any particular reason why splitting
up
the output from a query into multiple files is only relevant for binary
data.The files can be simply joined together outside psql
Just as multiple queries could be done to have the results put into
independent files.Personally I prefer relation type: single field, single file in special
g
command - because I can simply off all formatting and result should be
correct every time.Not sure why you think there would be a formatting issue or why the
result might not be 'correct'.Stephen, have you some use case for your request?
The initial patch forced a single value result. Including such a
restriction doesn't seem necessary to me. As for use-case, I've
certainly written code to work with binary-result data from PG
previously and it seems entirely reasonable that someone might wish to
pull data into a file with psql and then process it. I've been
wondering if we should consider how binary-mode COPY works, but that
format ends up being pretty inefficient due to the repeated 32-bit
length value for every field.My initial reaction was primairly that I didn't see value in the
somewhat arbitrary restriction being imposed on usage of this.
ok.
It is hard to design any solution - because there are not any intersection
on this basic simple things.
Regards
Pavel
Show quoted text
Thanks!
Stephen
On 3/18/17 12:51, Pavel Stehule wrote:
I rewrote these patches - it allows binary export/import from psql and
the code is very simple. The size of the patch is bigger due including
4KB binary file (in hex format 8KB).
This patch needs (at least) a rebase for the upcoming commit fest.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers