proposal: possibility to read dumped table's name from file

Started by Pavel Stehuleover 5 years ago207 messages
#1Pavel Stehule
pavel.stehule@gmail.com
1 attachment(s)

Hi

one my customer has to specify dumped tables name by name. After years and
increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from stdin).

I wrote simple PoC patch

Comments, notes, ideas?

Regards

Pavel

Attachments:

pg_dump-table-list-from-file.patchtext/x-patch; charset=US-ASCII; name=pg_dump-table-list-from-file.patchDownload
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bd14d38740..d8330ec32a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -362,6 +362,7 @@ main(int argc, char **argv)
 		{"serializable-deferrable", no_argument, &dopt.serializable_deferrable, 1},
 		{"snapshot", required_argument, NULL, 6},
 		{"strict-names", no_argument, &strict_names, 1},
+		{"table-names-from-file", required_argument, NULL, 8},
 		{"use-set-session-authorization", no_argument, &dopt.use_setsessauth, 1},
 		{"no-publications", no_argument, &dopt.no_publications, 1},
 		{"no-security-labels", no_argument, &dopt.no_security_labels, 1},
@@ -553,6 +554,59 @@ main(int argc, char **argv)
 				dosync = false;
 				break;
 
+			case 8:				/* read table names from file */
+				{
+					FILE	   *f;
+					char	   *line;
+					size_t		line_size = 1024;
+					ssize_t		chars;
+					bool		use_stdin = false;
+
+					if (strcmp(optarg, "-") != 0)
+					{
+						f = fopen(optarg, "r");
+						if (!f)
+						{
+							fprintf(stderr, _("%s: could not open the input file \"%s\": %s\n"),
+								progname, optarg, strerror(errno));
+							exit_nicely(1);
+						}
+					}
+					else
+					{
+						f = stdin;
+						use_stdin = true;
+					}
+
+					line = malloc(line_size);
+
+					while ((chars = getline(&line, &line_size, f)) != -1)
+					{
+						if (line[chars - 1] == '\n')
+							line[chars - 1] = '\0';
+
+						/* ignore empty rows */
+						if (*line != '\0')
+						{
+							simple_string_list_append(&table_include_patterns, line);
+							dopt.include_everything = false;
+						}
+					}
+
+					if (ferror(f))
+					{
+						fprintf(stderr, _("%s: could not read from file \"%s\": %s\n"),
+							progname, use_stdin ? "stdin" : optarg, strerror(errno));
+						exit_nicely(1);
+					}
+
+					if (!use_stdin)
+						fclose(f);
+
+					free(line);
+				}
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -977,6 +1031,8 @@ help(const char *progname)
 	printf(_("  --snapshot=SNAPSHOT          use given snapshot for the dump\n"));
 	printf(_("  --strict-names               require table and/or schema include patterns to\n"
 			 "                               match at least one entity each\n"));
+	printf(_("  --table-names-from-file=FILENAME\n"
+			 "                               read table's names from file (one table name per line)\n"));
 	printf(_("  --use-set-session-authorization\n"
 			 "                               use SET SESSION AUTHORIZATION commands instead of\n"
 			 "                               ALTER OWNER commands to set ownership\n"));
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Pavel Stehule (#1)
Re: proposal: possibility to read dumped table's name from file

Pavel Stehule <pavel.stehule@gmail.com> writes:

one my customer has to specify dumped tables name by name. After years and
increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from stdin).

I guess the question is why. That seems like an enormously error-prone
approach. Can't they switch to selecting schemas? Or excluding the
hopefully-short list of tables they don't want?

regards, tom lane

#3Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#2)
Re: proposal: possibility to read dumped table's name from file

pá 29. 5. 2020 v 16:28 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Pavel Stehule <pavel.stehule@gmail.com> writes:

one my customer has to specify dumped tables name by name. After years

and

increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from

stdin).

I guess the question is why. That seems like an enormously error-prone
approach. Can't they switch to selecting schemas? Or excluding the
hopefully-short list of tables they don't want?

It is not typical application. It is a analytic application when the schema
of database is based on dynamic specification of end user (end user can do
customization every time). So schema is very dynamic.

For example - typical server has about four thousand databases and every
database has some between 1K .. 10K tables.

Another specific are different versions of data in different tables. A user
can work with one set of data (one set of tables) and a application
prepares new set of data (new set of tables). Load can be slow, because
sometimes bigger tables are filled (about forty GB). pg_dump backups one
set of tables (little bit like snapshot of data). So it is strange OLAP
(but successfull) application.

Regards

Pavel

Show quoted text

regards, tom lane

#4David Fetter
david@fetter.org
In reply to: Pavel Stehule (#1)
Re: proposal: possibility to read dumped table's name from file

On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote:

Hi

one my customer has to specify dumped tables name by name. After years and
increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from stdin).

I wrote simple PoC patch

Comments, notes, ideas?

This seems like a handy addition. What I've done in cases similar to
this was to use `grep -f` on the output of `pg_dump -l` to create a
file suitable for `pg_dump -L`, or mash them together like this:

pg_restore -L <(pg_dump -l /path/to/dumpfile | grep -f /path/to/listfile) -d new_db /path/to/dumpfile

That's a lot of shell magic and obscure corners of commands to expect
people to use.

Would it make sense to expand this patch to handle other objects?

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: David Fetter (#4)
Re: proposal: possibility to read dumped table's name from file

David Fetter <david@fetter.org> writes:

Would it make sense to expand this patch to handle other objects?

If we're gonna do something like this, +1 for being more general.
The fact that pg_dump only has selection switches for tables and
schemas has always struck me as an omission.

regards, tom lane

#6Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#5)
Re: proposal: possibility to read dumped table's name from file

pá 29. 5. 2020 v 18:03 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

David Fetter <david@fetter.org> writes:

Would it make sense to expand this patch to handle other objects?

Sure. Just we should to design system (and names of options).

If we're gonna do something like this, +1 for being more general.
The fact that pg_dump only has selection switches for tables and
schemas has always struck me as an omission.

a implementation is trivial, hard is good design of names for these options.

Pavel

Show quoted text

regards, tom lane

#7Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#1)
Re: proposal: possibility to read dumped table's name from file

On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote:

one my customer has to specify dumped tables name by name. After years and
increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from stdin).

+1 - we would use this.

We put a regex (actually a pg_dump pattern) of tables to skip (timeseries
partitions which are older than a few days and which are also dumped once not
expected to change, and typically not redumped). We're nowhere near the
execve() limit, but it'd be nice if the command was primarily a list of options
and not a long regex.

Please also support reading from file for --exclude-table=pattern.

I'm drawing a parallel between this and rsync --include/--exclude and --filter.

We'd be implementing a new --filter, which might have similar syntax to rsync
(which I always forget).

--
Justin

#8Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#7)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

pá 29. 5. 2020 v 20:25 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote:

one my customer has to specify dumped tables name by name. After years

and

increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from

stdin).

+1 - we would use this.

We put a regex (actually a pg_dump pattern) of tables to skip (timeseries
partitions which are older than a few days and which are also dumped once
not
expected to change, and typically not redumped). We're nowhere near the
execve() limit, but it'd be nice if the command was primarily a list of
options
and not a long regex.

Please also support reading from file for --exclude-table=pattern.

I'm drawing a parallel between this and rsync --include/--exclude and
--filter.

We'd be implementing a new --filter, which might have similar syntax to
rsync
(which I always forget).

I implemented support for all "repeated" pg_dump options.

--exclude-schemas-file=FILENAME
--exclude-tables-data-file=FILENAME
--exclude-tables-file=FILENAME
--include-foreign-data-file=FILENAME
--include-schemas-file=FILENAME
--include-tables-file=FILENAME

Regards

Pavel

I invite any help with doc. There is just very raw text

Show quoted text

--
Justin

Attachments:

pg_dump-read-options-from-file.patchtext/x-patch; charset=US-ASCII; name=pg_dump-read-options-from-file.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 2f0807e912..5e39268d51 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -725,6 +725,16 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--exclude-schemas-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        This option allows to specify file with schemas that will not be
+        dumped. Any row of is one schema's name.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--exclude-table-data=<replaceable class="parameter">pattern</replaceable></option></term>
       <listitem>
@@ -743,6 +753,24 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--exclude-tables-data-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Do not dump data of tables spefified in file.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
+      <term><option>--exclude-tables-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Do not dump tables spefified in file.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--extra-float-digits=<replaceable class="parameter">ndigits</replaceable></option></term>
       <listitem>
@@ -795,6 +823,33 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--include-foreign-data-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Include data of foreign servers specified in file.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
+      <term><option>--include-schemas-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Dump schema(s) specified in file.
+       </para>
+      </listitem>
+     </varlistentry>
+
+     <varlistentry>
+      <term><option>--include-tables-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Dump table(s) specified in file.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--inserts</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dfe43968b8..db9fb10f68 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static bool read_options_from_file(SimpleStringList *slist, char *filename);
 
 
 int
@@ -362,8 +363,13 @@ main(int argc, char **argv)
 		{"disable-dollar-quoting", no_argument, &dopt.disable_dollar_quoting, 1},
 		{"disable-triggers", no_argument, &dopt.disable_triggers, 1},
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
+		{"exclude-schemas-file", required_argument, NULL, 12},
 		{"exclude-table-data", required_argument, NULL, 4},
+		{"exclude-tables-data-file", required_argument, NULL, 14},
+		{"exclude-tables-file", required_argument, NULL, 13},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"include-schemas-file", required_argument, NULL, 15},
+		{"include-tables-file", required_argument, NULL, 16},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -386,6 +392,7 @@ main(int argc, char **argv)
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -603,6 +610,32 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:				/* read exclude schama names from file */
+				(void) read_options_from_file(&schema_exclude_patterns, optarg);
+				break;
+
+			case 13:				/* read exclude table names from file */
+				(void) read_options_from_file(&table_exclude_patterns, optarg);
+				break;
+
+			case 14:				/* read exclude table data names from file */
+				(void) read_options_from_file(&tabledata_exclude_patterns, optarg);
+				break;
+
+			case 15:				/* read table names from file */
+				if (read_options_from_file(&schema_include_patterns, optarg))
+					dopt.include_everything = false;
+				break;
+
+			case 16:				/* read table names from file */
+				if (read_options_from_file(&table_include_patterns, optarg))
+					dopt.include_everything = false;
+				break;
+
+			case 17:				/* read exclude table data names from file */
+				(void) read_options_from_file(&foreign_servers_include_patterns, optarg);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1020,12 +1053,24 @@ help(const char *progname)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security (dump only content user has\n"
 			 "                               access to)\n"));
+	printf(_("  --exclude-schemas-file=FILENAME\n"
+			 "                               do NOT dump schema specified in file\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
+	printf(_("  --exclude-tables-data-file=FILENAME\n"
+			 "                               do NOT dump data for tables specified in file\n"));
+	printf(_("  --exclude-tables-file=FILENAME\n"
+			 "                               do NOT dump tables specified in file\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
 			 "                               servers matching PATTERN\n"));
+	printf(_("  --include-foreign-data-file=FILENAME\n"
+			 "                               include data of foreign servers specified by file\n"));
+	printf(_("  --include-schemas-file=FILENAME\n"
+			 "                               dump schema(s) specified in file\n"));
+	printf(_("  --include-tables-file=FILENAME\n"
+			 "                               dump table(s) specified in file\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
 	printf(_("  --no-comments                do not dump comments\n"));
@@ -18647,3 +18692,70 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Read list of values from file specified by name. Returns true when
+ * at least one value was read.
+ */
+static bool
+read_options_from_file(SimpleStringList *slist, char *filename)
+{
+	FILE	   *f;
+	char	   *line;
+	ssize_t		chars;
+	size_t		line_size = 1024;
+	bool		use_stdin = false;
+	bool		result = false;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		f = fopen(optarg, "r");
+		if (!f)
+		{
+			fprintf(stderr,
+					_("%s: could not open the input file \"%s\": %s\n"),
+					progname,
+					optarg,
+					strerror(errno));
+			exit_nicely(1);
+		}
+	}
+	else
+	{
+		f = stdin;
+		use_stdin = true;
+	}
+
+	line = malloc(line_size);
+
+	while ((chars = getline(&line, &line_size, f)) != -1)
+	{
+		if (line[chars - 1] == '\n')
+			line[chars - 1] = '\0';
+
+		/* ignore empty rows */
+		if (*line != '\0')
+		{
+			simple_string_list_append(slist, line);
+			result = true;
+		}
+	}
+
+	if (ferror(f))
+	{
+		fprintf(stderr,
+				_("%s: could not read from file \"%s\": %s\n"),
+				progname,
+				use_stdin ? "stdin" : optarg,
+				strerror(errno));
+		exit_nicely(1);
+	}
+
+	if (!use_stdin)
+		fclose(f);
+
+	free(line);
+
+	return result;
+}
#9Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#8)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jun 08, 2020 at 07:18:49PM +0200, Pavel Stehule wrote:

p� 29. 5. 2020 v 20:25 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote:

one my customer has to specify dumped tables name by name. After years and
increasing database size and table numbers he has problem with too short
command line. He need to read the list of tables from file (or from stdin).

+1 - we would use this.

We put a regex (actually a pg_dump pattern) of tables to skip (timeseries
partitions which are older than a few days and which are also dumped once not
expected to change, and typically not redumped). We're nowhere near the
execve() limit, but it'd be nice if the command was primarily a list of options
and not a long regex.

Please also support reading from file for --exclude-table=pattern.

I'm drawing a parallel between this and rsync --include/--exclude and
--filter.

We'd be implementing a new --filter, which might have similar syntax to rsync
(which I always forget).

I implemented support for all "repeated" pg_dump options.

I invite any help with doc. There is just very raw text

+ Do not dump data of tables spefified in file.

*specified

I still wonder if a better syntax would use a unified --filter option, whose
argument would allow including/excluding any type of object:

+[tn] include (t)table/(n)namespace/...
-[tn] exclude (t)table/(n)namespace/...

In the past, I looked for a way to exclude extended stats objects, and ended up
using a separate schema. An "extensible" syntax might be better (although
reading a file of just patterns has the advantage that the function can just be
called once for each option for each type of object).

--
Justin

#10Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#9)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

po 8. 6. 2020 v 23:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Mon, Jun 08, 2020 at 07:18:49PM +0200, Pavel Stehule wrote:

pá 29. 5. 2020 v 20:25 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote:

one my customer has to specify dumped tables name by name. After

years and

increasing database size and table numbers he has problem with too

short

command line. He need to read the list of tables from file (or from

stdin).

+1 - we would use this.

We put a regex (actually a pg_dump pattern) of tables to skip

(timeseries

partitions which are older than a few days and which are also dumped

once not

expected to change, and typically not redumped). We're nowhere near

the

execve() limit, but it'd be nice if the command was primarily a list

of options

and not a long regex.

Please also support reading from file for --exclude-table=pattern.

I'm drawing a parallel between this and rsync --include/--exclude and
--filter.

We'd be implementing a new --filter, which might have similar syntax

to rsync

(which I always forget).

I implemented support for all "repeated" pg_dump options.

I invite any help with doc. There is just very raw text

+ Do not dump data of tables spefified in file.

*specified

I am sending updated version - now with own implementation GNU (not POSIX)
function getline

I still wonder if a better syntax would use a unified --filter option, whose

argument would allow including/excluding any type of object:

+[tn] include (t)table/(n)namespace/...
-[tn] exclude (t)table/(n)namespace/...

In the past, I looked for a way to exclude extended stats objects, and
ended up
using a separate schema. An "extensible" syntax might be better (although
reading a file of just patterns has the advantage that the function can
just be
called once for each option for each type of object).

I tried to implement simple format "[+-][tndf] objectname"

please, check attached patch

Regards

Pavel

Show quoted text

--
Justin

Attachments:

pg_dump-filter.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 2f0807e912..332f0c312f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -813,6 +813,15 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read filters from file. Format "(+|-)(tnfd) objectname:
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--load-via-partition-root</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dfe43968b8..acc936ac7b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static size_t pg_getline(char **lineptr, size_t *n, FILE *fp);
+static void invalid_filter_format(char *filename, char *line, int lineno);
 
 
 int
@@ -364,6 +366,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +606,116 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				{
+					FILE	   *f;
+					char	   *line;
+					ssize_t		chars;
+					size_t		line_size = 1024;
+					int			lineno = 0;
+
+					/* use "-" as symbol for stdin */
+					if (strcmp(optarg, "-") != 0)
+					{
+						f = fopen(optarg, "r");
+						if (!f)
+							fatal("could not open the input file \"%s\": %s",
+								  optarg,
+								  strerror(errno));
+					}
+					else
+						f = stdin;
+
+					line = pg_malloc(line_size);
+
+					while ((chars = pg_getline(&line, &line_size, f)) != -1)
+					{
+						lineno += 1;
+
+						if (line[chars - 1] == '\n')
+							line[chars - 1] = '\0';
+
+						/* ignore empty rows */
+						if (*line != '\0')
+						{
+							bool		include_filter = false;
+							bool		exclude_filter = false;
+							char		objecttype;
+							char	   *objectname;
+
+							if (chars < 4)
+								invalid_filter_format(optarg, line, lineno);
+
+							if (line[0] == '+')
+								include_filter = true;
+							else if (line[0] == '-')
+								exclude_filter = true;
+							else
+								invalid_filter_format(optarg, line, lineno);
+
+							objecttype = line[1];
+
+							if (objecttype != 't' &&
+								objecttype != 'n' &&
+								objecttype != 'd' &&
+								objecttype != 'f')
+								invalid_filter_format(optarg, line, lineno);
+
+							objectname = &line[3];
+
+							/* skip initial spaces */
+							while (*objectname == ' ')
+								objectname++;
+
+							if (objecttype == 't')
+							{
+								if (include_filter)
+								{
+									simple_string_list_append(&table_include_patterns, objectname);
+									dopt.include_everything = false;
+								}
+								else if (exclude_filter)
+									simple_string_list_append(&table_exclude_patterns, objectname);
+							}
+							else if (objecttype == 'n')
+							{
+								if (include_filter)
+								{
+									simple_string_list_append(&schema_include_patterns, objectname);
+									dopt.include_everything = false;
+								}
+								else if (exclude_filter)
+									simple_string_list_append(&schema_exclude_patterns, objectname);
+							}
+							else if (objecttype == 'd')
+							{
+								if (include_filter)
+									invalid_filter_format(optarg, line, lineno);
+								else if (exclude_filter)
+									simple_string_list_append(&tabledata_exclude_patterns, objectname);
+							}
+							else if (objecttype == 'f')
+							{
+								if (include_filter)
+									simple_string_list_append(&foreign_servers_include_patterns, objectname);
+								else if (exclude_filter)
+									invalid_filter_format(optarg, line, lineno);
+							}
+						}
+					}
+
+					if (ferror(f))
+						fatal("could not read from file \"%s\": %s",
+							  f == stdin ? "stdin" : optarg,
+							  strerror(errno));
+
+					if (f != stdin)
+						fclose(f);
+
+					pg_free(line);
+				}
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1135,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object names from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18647,3 +18761,61 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += 1024;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+invalid_filter_format(char *filename, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\"",
+				 *filename == '-' ? "stdin" : filename);
+	fprintf(stderr, "%d: %s\n", lineno, line);
+	exit_nicely(1);
+}
#11Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#10)
Re: proposal: possibility to read dumped table's name from file

On Tue, Jun 09, 2020 at 11:46:24AM +0200, Pavel Stehule wrote:

po 8. 6. 2020 v 23:30 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

I still wonder if a better syntax would use a unified --filter option, whose

argument would allow including/excluding any type of object:

I tried to implement simple format "[+-][tndf] objectname"

Thanks.

+ /* ignore empty rows */
+ if (*line != '\0')

Maybe: if line=='\0': continue

We should also support comments.

+ bool include_filter = false;
+ bool exclude_filter = false;

I think we only need one bool.
You could call it: bool is_exclude = false

+
+							if (chars < 4)
+								invalid_filter_format(optarg, line, lineno);

I think that check is too lax.
I think it's ok if we require the first char to be [-+] and the 2nd char to be
[dntf]

+ objecttype = line[1];

... but I think this is inadequately "liberal in what it accepts"; I think it
should skip spaces. In my proposed scheme, someone might reasonably write:

+
+							objectname = &line[3];
+
+							/* skip initial spaces */
+							while (*objectname == ' ')
+								objectname++;

I suggest to use isspace()

I think we should check that *objectname != '\0', rather than chars>=4, above.

+								if (include_filter)
+								{
+									simple_string_list_append(&table_include_patterns, objectname);
+									dopt.include_everything = false;
+								}
+								else if (exclude_filter)
+									simple_string_list_append(&table_exclude_patterns, objectname);

If you use bool is_exclude, then this becomes "else" and you don't need to
think about checking if (!include && !exclude).

+							else if (objecttype == 'f')
+							{
+								if (include_filter)
+									simple_string_list_append(&foreign_servers_include_patterns, objectname);
+								else if (exclude_filter)
+									invalid_filter_format(optarg, line, lineno);
+							}

I would handle invalid object types as "else: invalid_filter_format()" here,
rather than duplicating above as: !=ALL('d','n','t','f')

+
+					if (ferror(f))
+						fatal("could not read from file \"%s\": %s",
+							  f == stdin ? "stdin" : optarg,
+							  strerror(errno));

I think we're allowed to use %m here ?

+ printf(_(" --filter=FILENAME read object names from file\n"));

Object name filter expression, or something..

+ * getline is originaly GNU function, and should not be everywhere still.

originally

+ * Use own reduced implementation.

Did you "reduce" this from another implementation? Where?
What is its license ?

Maybe a line-reader already exists in the frontend (?) .. or maybe it should.

--
Justin

#12Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#11)
Re: proposal: possibility to read dumped table's name from file

st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Tue, Jun 09, 2020 at 11:46:24AM +0200, Pavel Stehule wrote:

po 8. 6. 2020 v 23:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

I still wonder if a better syntax would use a unified --filter option,

whose

argument would allow including/excluding any type of object:

I tried to implement simple format "[+-][tndf] objectname"

I had another idea about format - instead using +-, we can use case
sensitive options same to pg_dump command line (with extending Df -
because these options doesn't exists in short form)

So format can looks like

[tTnNDf] {objectname}

What do you think about this? This format is simpler, and it can work. What
do you think about it?

Did you "reduce" this from another implementation? Where?
What is its license ?

The code is 100% mine. It is not copy from gnulib and everybody can simply
check it

https://code.woboq.org/userspace/glibc/stdio-common/getline.c.html
https://code.woboq.org/userspace/glibc/libio/iogetdelim.c.html#_IO_getdelim

Reduced in functionality sense. There is no full argument check that is
necessary for glibc functions. There are no memory checks because
pg_malloc, pg_realloc are used.

#13Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#12)
Re: proposal: possibility to read dumped table's name from file

On Wed, Jun 10, 2020 at 05:03:49AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

On Tue, Jun 09, 2020 at 11:46:24AM +0200, Pavel Stehule wrote:

po 8. 6. 2020 v 23:30 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

I still wonder if a better syntax would use a unified --filter option, whose

argument would allow including/excluding any type of object:

I tried to implement simple format "[+-][tndf] objectname"

I had another idea about format - instead using +-, we can use case
sensitive options same to pg_dump command line (with extending Df -
because these options doesn't exists in short form)

So format can looks like

[tTnNDf] {objectname}

What do you think about this? This format is simpler, and it can work. What
do you think about it?

I prefer [-+][dtnf], which is similar to rsync --filter, and clear what it's
doing. I wouldn't put much weight on what the short options are.

I wonder if some people would want to be able to use *long* or short options:

-table foo
+schema baz

Or maybe:

exclude-table=foo
schema=bar

Some tools use "long options without leading dashes" as their configuration
file format. Examples: openvpn, mysql. So that could be a good option.
OTOH, there's only a few "keys", so I'm not sure how many people would want to
repeat them, if there's enough to bother putting them in the file rather than
the cmdline.

--
Justin

#14Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#11)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Tue, Jun 09, 2020 at 11:46:24AM +0200, Pavel Stehule wrote:

po 8. 6. 2020 v 23:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

I still wonder if a better syntax would use a unified --filter option,

whose

argument would allow including/excluding any type of object:

I tried to implement simple format "[+-][tndf] objectname"

Thanks.

+                                             /* ignore empty rows */
+                                             if (*line != '\0')

Maybe: if line=='\0': continue

ok

We should also support comments.

+ bool

include_filter = false;

+ bool

exclude_filter = false;

I think we only need one bool.
You could call it: bool is_exclude = false

ok

+

+                                                     if (chars < 4)
+

invalid_filter_format(optarg, line, lineno);

I think that check is too lax.
I think it's ok if we require the first char to be [-+] and the 2nd char
to be
[dntf]

+ objecttype =

line[1];

... but I think this is inadequately "liberal in what it accepts"; I think
it
should skip spaces. In my proposed scheme, someone might reasonably write:

+
+                                                     objectname =

&line[3];

+
+                                                     /* skip initial

spaces */

+ while (*objectname

== ' ')

+

objectname++;

I suggest to use isspace()

ok

I think we should check that *objectname != '\0', rather than chars>=4,
above.

done

+ if

(include_filter)

+                                                             {
+

simple_string_list_append(&table_include_patterns, objectname);

+

dopt.include_everything = false;

+                                                             }
+                                                             else if

(exclude_filter)

+

simple_string_list_append(&table_exclude_patterns, objectname);

If you use bool is_exclude, then this becomes "else" and you don't need to
think about checking if (!include && !exclude).

+ else if

(objecttype == 'f')

+                                                     {
+                                                             if

(include_filter)

+

simple_string_list_append(&foreign_servers_include_patterns, objectname);

+ else if

(exclude_filter)

+

invalid_filter_format(optarg, line, lineno);

+ }

I would handle invalid object types as "else: invalid_filter_format()"
here,
rather than duplicating above as: !=ALL('d','n','t','f')

good idea

+
+                                     if (ferror(f))
+                                             fatal("could not read from

file \"%s\": %s",

+ f == stdin ?

"stdin" : optarg,

+ strerror(errno));

I think we're allowed to use %m here ?

changed

+ printf(_(" --filter=FILENAME read object names from

file\n"));

Object name filter expression, or something..

yes, it is not object names now

+ * getline is originaly GNU function, and should not be everywhere

still.
originally

+ * Use own reduced implementation.

Did you "reduce" this from another implementation? Where?
What is its license ?

Maybe a line-reader already exists in the frontend (?) .. or maybe it
should.

everywhere else is used a function fgets. Currently pg_getline is used just
on only one place, so I don't think so moving it to some common part is
maybe premature.

Maybe it can be used as replacement of some fgets calls, but then it is
different topic, I think.

Thank you for comments, attached updated patch

Regards

Pavel

Show quoted text

--
Justin

Attachments:

pg_dump-filter-20200611.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200611.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 2f0807e912..332f0c312f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -813,6 +813,15 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read filters from file. Format "(+|-)(tnfd) objectname:
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--load-via-partition-root</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dfe43968b8..9e76189635 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static size_t pg_getline(char **lineptr, size_t *n, FILE *fp);
+static void invalid_filter_format(char *message, char *filename, char *line, int lineno);
 
 
 int
@@ -364,6 +366,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +606,135 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				{
+					FILE	   *f;
+					char	   *line;
+					ssize_t		chars;
+					size_t		line_size = 1024;
+					int			lineno = 0;
+
+					/* use "-" as symbol for stdin */
+					if (strcmp(optarg, "-") != 0)
+					{
+						f = fopen(optarg, "r");
+						if (!f)
+							fatal("could not open the input file \"%s\": %m",
+								  optarg);
+					}
+					else
+						f = stdin;
+
+					line = pg_malloc(line_size);
+
+					while ((chars = pg_getline(&line, &line_size, f)) != -1)
+					{
+						bool		is_include;
+						char		objecttype;
+						char	   *objectname;
+
+						lineno += 1;
+
+						if (line[chars - 1] == '\n')
+							line[chars - 1] = '\0';
+
+						/* ignore empty rows */
+						if (*line == '\0')
+							continue;
+
+						if (chars < 2)
+							invalid_filter_format("too short line",
+												  optarg,
+												  line,
+												  lineno);
+
+						if (line[0] == '+')
+							is_include = true;
+						else if (line[0] == '-')
+							is_include = false;
+						else
+							invalid_filter_format("invalid option type (use [+-]",
+												  optarg,
+												  line,
+												  lineno);
+
+						objecttype = line[1];
+						objectname = &line[2];
+
+						/* skip initial spaces */
+						while (isspace(*objectname))
+							objectname++;
+
+						if (*objectname == '\0')
+							invalid_filter_format("missing object name",
+												  optarg,
+												  line,
+												  lineno);
+
+						if (objecttype == 't')
+						{
+							if (is_include)
+							{
+								simple_string_list_append(&table_include_patterns,
+														  objectname);
+								dopt.include_everything = false;
+							}
+							else
+								simple_string_list_append(&table_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'n')
+						{
+							if (is_include)
+							{
+								simple_string_list_append(&schema_include_patterns,
+														  objectname);
+								dopt.include_everything = false;
+							}
+							else
+								simple_string_list_append(&schema_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'd')
+						{
+							if (is_include)
+								invalid_filter_format("include filter is not supported for this type of object",
+													  optarg,
+													  line,
+													  lineno);
+							else
+								simple_string_list_append(&tabledata_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'f')
+						{
+							if (is_include)
+								simple_string_list_append(&foreign_servers_include_patterns,
+														  objectname);
+							else
+								invalid_filter_format("exclude filter is not supported for this type of object",
+													  optarg,
+													  line,
+													  lineno);
+						}
+						else
+							invalid_filter_format("invalid object type (use [tndf])",
+												  optarg,
+												  line,
+												  lineno);
+					}
+
+					if (ferror(f))
+						fatal("could not read from file \"%s\": %m",
+							  f == stdin ? "stdin" : optarg);
+
+					if (f != stdin)
+						fclose(f);
+
+					pg_free(line);
+				}
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1154,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object name filter expressions from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18647,3 +18780,67 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += 1024;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+invalid_filter_format(char *message, char *filename, char *line, int lineno)
+{
+	char	   *displayname;
+
+	displayname = *filename == '-' ? "stdin" : filename;
+
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 displayname,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+	exit_nicely(1);
+}
#15vignesh C
vignesh21@gmail.com
In reply to: Pavel Stehule (#14)
Re: proposal: possibility to read dumped table's name from file

On Thu, Jun 11, 2020 at 1:07 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

Thank you for comments, attached updated patch

Few comments:
+invalid_filter_format(char *message, char *filename, char *line, int lineno)
+{
+       char       *displayname;
+
+       displayname = *filename == '-' ? "stdin" : filename;
+
+       pg_log_error("invalid format of filter file \"%s\": %s",
+                                displayname,
+                                message);
+
+       fprintf(stderr, "%d: %s\n", lineno, line);
+       exit_nicely(1);
+}
I think fclose is missing here.
+                                               if (line[chars - 1] == '\n')
+                                                       line[chars - 1] = '\0';
Should we check for '\r' also to avoid failures in some platforms.
+     <varlistentry>
+      <term><option>--filter=<replaceable
class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read filters from file. Format "(+|-)(tnfd) objectname:
+       </para>
+      </listitem>
+     </varlistentry>

I felt some documentation is missing here. We could include,
options tnfd is for controlling table, schema, foreign server data &
table exclude patterns.

Instead of using tnfd, if we could use the same options as existing
pg_dump options it will be less confusing.

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

#16Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#14)
Re: proposal: possibility to read dumped table's name from file

On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

+                                             /* ignore empty rows */
+                                             if (*line != '\0')

Maybe: if line=='\0': continue
We should also support comments.

Comment support is still missing but easily added :)

I tried this patch and it works for my purposes.

Also, your getline is dynamically re-allocating lines of arbitrary length.
Possibly that's not needed. We'll typically read "+t schema.relname", which is
132 chars. Maybe it's sufficient to do
char buf[1024];
fgets(buf);
if strchr(buf, '\n') == NULL: error();
ret = pstrdup(buf);

In any case, you could have getline return a char* and (rather than following
GNU) no need to take char**, int* parameters to conflate inputs and outputs.

I realized that --filter has an advantage over the previous implementation
(with multiple --exclude-* and --include-*) in that it's possible to use stdin
for includes *and* excludes.

By chance, I had the opportunity yesterday to re-use with rsync a regex that
I'd previously been using with pg_dump and grep. What this patch calls
"--filter" in rsync is called "--filter-from". rsync's --filter-from rejects
filters of length longer than max filename, so I had to split it up into
multiple lines instead of using regex alternation ("|"). This option is a
close parallel in pg_dump.

--
Justin

#17Pavel Stehule
pavel.stehule@gmail.com
In reply to: vignesh C (#15)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

so 27. 6. 2020 v 14:55 odesílatel vignesh C <vignesh21@gmail.com> napsal:

On Thu, Jun 11, 2020 at 1:07 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Thank you for comments, attached updated patch

Few comments:
+invalid_filter_format(char *message, char *filename, char *line, int
lineno)
+{
+       char       *displayname;
+
+       displayname = *filename == '-' ? "stdin" : filename;
+
+       pg_log_error("invalid format of filter file \"%s\": %s",
+                                displayname,
+                                message);
+
+       fprintf(stderr, "%d: %s\n", lineno, line);
+       exit_nicely(1);
+}

I think fclose is missing here.

done

+                                               if (line[chars - 1] ==
'\n')
+                                                       line[chars - 1] =
'\0';
Should we check for '\r' also to avoid failures in some platforms.

I checked other usage of fgets in Postgres source code, and everywhere is
used test on \n

When I did some fast research, then
https://stackoverflow.com/questions/12769289/carriage-return-by-fgets \r in
this case should be thrown by libc on Microsoft

https://stackoverflow.com/questions/2061334/fgets-linux-vs-mac

\n should be on Mac OS X .. 2001 year .. I am not sure if Mac OS 9 should
be supported.

+     <varlistentry>
+      <term><option>--filter=<replaceable
class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read filters from file. Format "(+|-)(tnfd) objectname:
+       </para>
+      </listitem>
+     </varlistentry>

I felt some documentation is missing here. We could include,
options tnfd is for controlling table, schema, foreign server data &
table exclude patterns.

I have a plan to completate doc when the design is completed. It was not
clear if people prefer long or short forms of option names.

Instead of using tnfd, if we could use the same options as existing
pg_dump options it will be less confusing.

it almost same

+-t .. tables
+-n schema
-d exclude data .. there is not short option for --exclude-table-data
+f include foreign table .. there is not short option for
--include-foreign-data

So still, there is a opened question if use +-tnfd system, or system based
on long option

table foo
exclude-table foo
schema xx
exclude-schema xx
include-foreign-data yyy
exclude-table-data zzz

Typically these files will be generated by scripts and processed via pipe,
so there I see just two arguments for and aginst:

short format - there is less probability to do typo error (but there is not
full consistency with pg_dump options)
long format - it is self documented (and there is full consistency with
pg_dump)

In this case I prefer short form .. it is more comfortable for users, and
there are only a few variants, so it is not necessary to use too verbose
language (design). But my opinion is not aggressively strong and I'll
accept any common agreement.

Regards

Updated patch attached

Show quoted text

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

Attachments:

pg_dump-filter-20200705.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200705.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 2f0807e912..332f0c312f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -813,6 +813,15 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read filters from file. Format "(+|-)(tnfd) objectname:
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--load-via-partition-root</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a41a3db876..a3f8bebf52 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static size_t pg_getline(char **lineptr, size_t *n, FILE *fp);
+static void exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno);
 
 
 int
@@ -364,6 +366,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +606,141 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				{
+					FILE	   *f;
+					char	   *line;
+					ssize_t		chars;
+					size_t		line_size = 1024;
+					int			lineno = 0;
+
+					/* use "-" as symbol for stdin */
+					if (strcmp(optarg, "-") != 0)
+					{
+						f = fopen(optarg, "r");
+						if (!f)
+							fatal("could not open the input file \"%s\": %m",
+								  optarg);
+					}
+					else
+						f = stdin;
+
+					line = pg_malloc(line_size);
+
+					while ((chars = pg_getline(&line, &line_size, f)) != -1)
+					{
+						bool		is_include;
+						char		objecttype;
+						char	   *objectname;
+
+						lineno += 1;
+
+						if (line[chars - 1] == '\n')
+							line[chars - 1] = '\0';
+
+						/* ignore empty rows */
+						if (*line == '\0')
+							continue;
+
+						if (chars < 2)
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "too short line",
+													   line,
+													   lineno);
+
+						if (line[0] == '+')
+							is_include = true;
+						else if (line[0] == '-')
+							is_include = false;
+						else
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "invalid option type (use [+-]",
+													   line,
+													   lineno);
+
+						objecttype = line[1];
+						objectname = &line[2];
+
+						/* skip initial spaces */
+						while (isspace(*objectname))
+							objectname++;
+
+						if (*objectname == '\0')
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "missing object name",
+													   line,
+													   lineno);
+
+						if (objecttype == 't')
+						{
+							if (is_include)
+							{
+								simple_string_list_append(&table_include_patterns,
+														  objectname);
+								dopt.include_everything = false;
+							}
+							else
+								simple_string_list_append(&table_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'n')
+						{
+							if (is_include)
+							{
+								simple_string_list_append(&schema_include_patterns,
+														  objectname);
+								dopt.include_everything = false;
+							}
+							else
+								simple_string_list_append(&schema_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'd')
+						{
+							if (is_include)
+								exit_invalid_filter_format(f,
+														   optarg,
+														   "include filter is not supported for this type of object",
+														   line,
+														   lineno);
+							else
+								simple_string_list_append(&tabledata_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'f')
+						{
+							if (is_include)
+								simple_string_list_append(&foreign_servers_include_patterns,
+														  objectname);
+							else
+								exit_invalid_filter_format(f,
+														   optarg,
+														   "exclude filter is not supported for this type of object",
+														   line,
+														   lineno);
+						}
+						else
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "invalid object type (use [tndf])",
+													   line,
+													   lineno);
+					}
+
+					if (ferror(f))
+						fatal("could not read from file \"%s\": %m",
+							  f == stdin ? "stdin" : optarg);
+
+					if (f != stdin)
+						fclose(f);
+
+					pg_free(line);
+				}
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1160,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object name filter expressions from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18624,3 +18763,67 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += 1024;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 *filename == '-' ? "stdin" : filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
#18Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#16)
Re: proposal: possibility to read dumped table's name from file

st 1. 7. 2020 v 23:24 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

+                                             /* ignore empty rows */
+                                             if (*line != '\0')

Maybe: if line=='\0': continue
We should also support comments.

Comment support is still missing but easily added :)

I tried this patch and it works for my purposes.

Also, your getline is dynamically re-allocating lines of arbitrary length.
Possibly that's not needed. We'll typically read "+t schema.relname",
which is
132 chars. Maybe it's sufficient to do
char buf[1024];
fgets(buf);
if strchr(buf, '\n') == NULL: error();
ret = pstrdup(buf);

63 bytes is max effective identifier size, but it is not max size of
identifiers. It is very probably so buff with 1024 bytes will be enough for
all, but I do not want to increase any new magic limit. More when dynamic
implementation is not too hard.

Table name can be very long - sometimes the data names (table names) can be
stored in external storages with full length and should not be practical to
require truncating in filter file.

For this case it is very effective, because a resized (increased) buffer is
used for following rows, so realloc should not be often. So when I have to
choose between two implementations with similar complexity, I prefer more
dynamic code without hardcoded limits. This dynamic hasn't any overhead.

In any case, you could have getline return a char* and (rather than
following
GNU) no need to take char**, int* parameters to conflate inputs and
outputs.

no, it has a special benefit. It eliminates the short malloc/free cycle.
When some lines are longer, then the buffer is increased (and limits), and
for other rows with same or less size is not necessary realloc.

I realized that --filter has an advantage over the previous implementation
(with multiple --exclude-* and --include-*) in that it's possible to use
stdin
for includes *and* excludes.

yes, it looks like better choose

By chance, I had the opportunity yesterday to re-use with rsync a regex
that
I'd previously been using with pg_dump and grep. What this patch calls
"--filter" in rsync is called "--filter-from". rsync's --filter-from
rejects
filters of length longer than max filename, so I had to split it up into
multiple lines instead of using regex alternation ("|"). This option is a
close parallel in pg_dump.

we can talk about option name - maybe "--filter-from" is better than just
"--filter"

Regards

Pavel

Show quoted text

--
Justin

#19Justin Pryzby
pryzby@telsasoft.com
In reply to: Justin Pryzby (#16)
Re: proposal: possibility to read dumped table's name from file

On Wed, Jul 01, 2020 at 04:24:52PM -0500, Justin Pryzby wrote:

On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

+                                             /* ignore empty rows */
+                                             if (*line != '\0')

Maybe: if line=='\0': continue
We should also support comments.

Comment support is still missing but easily added :)

Still missing from the latest patch.

With some added documentation, I think this can be RfC.

--
Justin

#20Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#19)
Re: proposal: possibility to read dumped table's name from file

ne 5. 7. 2020 v 22:31 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Wed, Jul 01, 2020 at 04:24:52PM -0500, Justin Pryzby wrote:

On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

+ /* ignore empty rows

*/

+ if (*line != '\0')

Maybe: if line=='\0': continue
We should also support comments.

Comment support is still missing but easily added :)

Still missing from the latest patch.

I can implement a comment support. But I am not sure about the format. The
start can be "--" or classic #.

but "--" can be in this context messy

Show quoted text

With some added documentation, I think this can be RfC.

--
Justin

#21Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#20)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

ne 5. 7. 2020 v 22:37 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

ne 5. 7. 2020 v 22:31 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Wed, Jul 01, 2020 at 04:24:52PM -0500, Justin Pryzby wrote:

On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

+ /* ignore empty

rows */

+ if (*line != '\0')

Maybe: if line=='\0': continue
We should also support comments.

Comment support is still missing but easily added :)

Still missing from the latest patch.

I can implement a comment support. But I am not sure about the format. The
start can be "--" or classic #.

but "--" can be in this context messy

here is support for comment's line - first char should be #

Regards

Pavel

Show quoted text

With some added documentation, I think this can be RfC.

--
Justin

Attachments:

pg_dump-filter-20200706.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200706.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 2f0807e912..332f0c312f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -813,6 +813,15 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read filters from file. Format "(+|-)(tnfd) objectname:
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--load-via-partition-root</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a41a3db876..0b16d528c2 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static size_t pg_getline(char **lineptr, size_t *n, FILE *fp);
+static void exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno);
 
 
 int
@@ -364,6 +366,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +606,145 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				{
+					FILE	   *f;
+					char	   *line;
+					ssize_t		chars;
+					size_t		line_size = 1024;
+					int			lineno = 0;
+
+					/* use "-" as symbol for stdin */
+					if (strcmp(optarg, "-") != 0)
+					{
+						f = fopen(optarg, "r");
+						if (!f)
+							fatal("could not open the input file \"%s\": %m",
+								  optarg);
+					}
+					else
+						f = stdin;
+
+					line = pg_malloc(line_size);
+
+					while ((chars = pg_getline(&line, &line_size, f)) != -1)
+					{
+						bool		is_include;
+						char		objecttype;
+						char	   *objectname;
+
+						lineno += 1;
+
+						if (line[chars - 1] == '\n')
+							line[chars - 1] = '\0';
+
+						/* ignore empty rows */
+						if (*line == '\0')
+							continue;
+
+						/* when first char is hash, ignore whole line */
+						if (*line == '#')
+							continue;
+
+						if (chars < 2)
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "too short line",
+													   line,
+													   lineno);
+
+						if (line[0] == '+')
+							is_include = true;
+						else if (line[0] == '-')
+							is_include = false;
+						else
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "invalid option type (use [+-]",
+													   line,
+													   lineno);
+
+						objecttype = line[1];
+						objectname = &line[2];
+
+						/* skip initial spaces */
+						while (isspace(*objectname))
+							objectname++;
+
+						if (*objectname == '\0')
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "missing object name",
+													   line,
+													   lineno);
+
+						if (objecttype == 't')
+						{
+							if (is_include)
+							{
+								simple_string_list_append(&table_include_patterns,
+														  objectname);
+								dopt.include_everything = false;
+							}
+							else
+								simple_string_list_append(&table_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'n')
+						{
+							if (is_include)
+							{
+								simple_string_list_append(&schema_include_patterns,
+														  objectname);
+								dopt.include_everything = false;
+							}
+							else
+								simple_string_list_append(&schema_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'd')
+						{
+							if (is_include)
+								exit_invalid_filter_format(f,
+														   optarg,
+														   "include filter is not supported for this type of object",
+														   line,
+														   lineno);
+							else
+								simple_string_list_append(&tabledata_exclude_patterns,
+														  objectname);
+						}
+						else if (objecttype == 'f')
+						{
+							if (is_include)
+								simple_string_list_append(&foreign_servers_include_patterns,
+														  objectname);
+							else
+								exit_invalid_filter_format(f,
+														   optarg,
+														   "exclude filter is not supported for this type of object",
+														   line,
+														   lineno);
+						}
+						else
+							exit_invalid_filter_format(f,
+													   optarg,
+													   "invalid object type (use [tndf])",
+													   line,
+													   lineno);
+					}
+
+					if (ferror(f))
+						fatal("could not read from file \"%s\": %m",
+							  f == stdin ? "stdin" : optarg);
+
+					if (f != stdin)
+						fclose(f);
+
+					pg_free(line);
+				}
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1164,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object name filter expressions from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18624,3 +18767,67 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += 1024;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 *filename == '-' ? "stdin" : filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
#22Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#21)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jul 06, 2020 at 06:34:15AM +0200, Pavel Stehule wrote:

Comment support is still missing but easily added :)

Still missing from the latest patch.

I can implement a comment support. But I am not sure about the format. The

here is support for comment's line - first char should be #

Thanks, that's how I assumed it would look.

With some added documentation, I think this can be RfC.

Do you want to add any more documentation ?

Few more things:

+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 *filename == '-' ? "stdin" : filename,
+				 message);

You refer to as "stdin" any filename beginning with -.

I think you could just print "-" and not "stdin".
In any case, *filename=='-' is wrong since it only checks filename[0].
In a few places you compare ==stdin (which is right).

Also, I think "f" isn't as good a name as "fp".

You're adding 139 lines to a 600 line main(), and no other option is more than
15 lines. Would you put it in a separate function ?

--
Justin

#23vignesh C
vignesh21@gmail.com
In reply to: Pavel Stehule (#21)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jul 6, 2020 at 10:05 AM Pavel Stehule <pavel.stehule@gmail.com> wrote:

here is support for comment's line - first char should be #

Few comments:
+               str = fgets(*lineptr + total_chars,
+                                       *n - total_chars,
+                                       fp);
+
+               if (ferror(fp))
+                       return -1;

Should we include any error message in the above case.

+               else
+                       break;
+       }
+
+       if (ferror(fp))
+               return -1;

Similar to above.

+                       /* check, if there is good enough space for
next content */
+                       if (*n - total_chars < 2)
+                       {
+                               *n += 1024;
+                               *lineptr = pg_realloc(*lineptr, *n);
+                       }
We could use a macro in place of 1024.
+                                               if (objecttype == 't')
+                                               {
+                                                       if (is_include)
+                                                       {
+
simple_string_list_append(&table_include_patterns,
+
                                           objectname);
+
dopt.include_everything = false;
+                                                       }
+                                                       else
+
simple_string_list_append(&table_exclude_patterns,
+
                                           objectname);
+                                               }
+                                               else if (objecttype == 'n')
+                                               {
+                                                       if (is_include)
+                                                       {
+
simple_string_list_append(&schema_include_patterns,
+
                                           objectname);
+
dopt.include_everything = false;
+                                                       }
+                                                       else
+
simple_string_list_append(&schema_exclude_patterns,
+
                                           objectname);
+                                               }
Some of the above code is repetitive in above, can the common code be
made into a macro and called?

printf(_(" --extra-float-digits=NUM override default
setting for extra_float_digits\n"));
+ printf(_(" --filter=FILENAME read object name
filter expressions from file\n"));
printf(_(" --if-exists use IF EXISTS when
dropping objects\n"));
Can this be changed to dump objects and data based on the filter
expressions from the filter file.

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

#24Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#22)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

ne 12. 7. 2020 v 1:06 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Mon, Jul 06, 2020 at 06:34:15AM +0200, Pavel Stehule wrote:

Comment support is still missing but easily added :)

Still missing from the latest patch.

I can implement a comment support. But I am not sure about the format.

The

here is support for comment's line - first char should be #

Thanks, that's how I assumed it would look.

With some added documentation, I think this can be RfC.

Do you want to add any more documentation ?

done

Few more things:

+exit_invalid_filter_format(FILE *fp, char *filename, char *message,

char *line, int lineno)

+{
+     pg_log_error("invalid format of filter file \"%s\": %s",
+                              *filename == '-' ? "stdin" : filename,
+                              message);

You refer to as "stdin" any filename beginning with -.

I think you could just print "-" and not "stdin".
In any case, *filename=='-' is wrong since it only checks filename[0].
In a few places you compare ==stdin (which is right).

done

Also, I think "f" isn't as good a name as "fp".

done

You're adding 139 lines to a 600 line main(), and no other option is more
than
15 lines. Would you put it in a separate function ?

done

please, check attached patch

Regards

Pavel

Show quoted text

--
Justin

Attachments:

pg_dump-filter-20200713-1.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200713-1.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7a37fd8045..2f2bfb4dbf 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,99 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        This option ensure reading object's filters from specified file.
+        If you use "-" as filename, then stdin is used as source. This file
+        has to have following line format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+        Only one object name can be specified per one line:
+<programlisting>
++t mytable1
++t mytable2
++f some_foreign_table
+-d mytable3
+</programlisting>
+        With this file the dump ensures dump table <literal>mytable1</literal>,
+        <literal>mytable2</literal>. The data of foreign table
+        <literal>some_foreign_table</literal> will be dumped too. And the data
+        of <literal>mytable3</literal> will not be dumped.
+       </para>
+
+       <para>
+        The first char <literal>+</literal> or <literal>-</literal> specifies
+        if object name will be used as include or exclude filter.
+       </para>
+
+       <para>
+        The second char
+        <literal>t</literal>,
+        <literal>n</literal>,
+        <literal>f</literal>,
+        <literal>d</literal>
+        specifies a object type.
+
+        <variablelist>
+         <varlistentry>
+          <term><literal>t</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--table</option>. In exclusive form (<literal>-</literal>)
+            it is same like <option>--exclude-table</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>n</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--schema</option>. In exclusive form (<literal>-</literal>)
+            it is same like <option>--exclude-schema</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>f</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--include-foreign-data</option>. The exclusive form
+            (<literal>-</literal>) is not allowed.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>d</literal></term>
+          <listitem>
+           <para>
+            The inclusive form (<literal>+</literal>) is not allowed.
+            In exclusive form (<literal>-</literal>) it is same like
+            <option>--exclude-table-data</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+        </variablelist>
+       </para>
+
+       <para>
+        The option <option>--filter</option> can be used together with options
+        <option>--table</option>, <option>--exclude-table</option>,
+        <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--include-foreign-data</option> and
+        <option>--exclude-table-data</option>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e758b5c50a..2fe6db9b05 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -364,6 +365,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +605,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1028,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object name filter expressions from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18597,3 +18604,208 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += 1024;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	char	   *line;
+	ssize_t		chars;
+	size_t		line_size = 1024;
+	int			lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	line = pg_malloc(line_size);
+
+	while ((chars = pg_getline(&line, &line_size, fp)) != -1)
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+
+		lineno += 1;
+
+		if (line[chars - 1] == '\n')
+			line[chars - 1] = '\0';
+
+		/* ignore empty rows */
+		if (*line == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*line == '#')
+			continue;
+
+		if (chars < 2)
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "too short line",
+									   line,
+									   lineno);
+
+		if (line[0] == '+')
+			is_include = true;
+		else if (line[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   line,
+									   lineno);
+
+		objecttype = line[1];
+		objectname = &line[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   line,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   line,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   line,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   line,
+									   lineno);
+	}
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	pg_free(line);
+}
#25Pavel Stehule
pavel.stehule@gmail.com
In reply to: vignesh C (#23)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

ne 12. 7. 2020 v 3:43 odesílatel vignesh C <vignesh21@gmail.com> napsal:

On Mon, Jul 6, 2020 at 10:05 AM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

here is support for comment's line - first char should be #

Few comments:
+               str = fgets(*lineptr + total_chars,
+                                       *n - total_chars,
+                                       fp);
+
+               if (ferror(fp))
+                       return -1;

Should we include any error message in the above case.

+               else
+                       break;
+       }
+
+       if (ferror(fp))
+               return -1;

Similar to above.

it should be ok, both variant finishing by

<-->if (ferror(fp))
<--><-->fatal("could not read from file \"%s\": %m", filename);

%m should to print related error message

+                       /* check, if there is good enough space for
next content */
+                       if (*n - total_chars < 2)
+                       {
+                               *n += 1024;
+                               *lineptr = pg_realloc(*lineptr, *n);
+                       }
We could use a macro in place of 1024.

done

+                                               if (objecttype == 't')
+                                               {
+                                                       if (is_include)
+                                                       {
+
simple_string_list_append(&table_include_patterns,
+
objectname);
+
dopt.include_everything = false;
+                                                       }
+                                                       else
+
simple_string_list_append(&table_exclude_patterns,
+
objectname);
+                                               }
+                                               else if (objecttype == 'n')
+                                               {
+                                                       if (is_include)
+                                                       {
+
simple_string_list_append(&schema_include_patterns,
+
objectname);
+
dopt.include_everything = false;
+                                                       }
+                                                       else
+
simple_string_list_append(&schema_exclude_patterns,
+
objectname);
+                                               }
Some of the above code is repetitive in above, can the common code be
made into a macro and called?

There are two same fragments and two different fragments. In this case I
don't think so using macro or auxiliary function can help with readability.
Current code is well structured and well readable.

printf(_(" --extra-float-digits=NUM override default
setting for extra_float_digits\n"));
+ printf(_(" --filter=FILENAME read object name
filter expressions from file\n"));
printf(_(" --if-exists use IF EXISTS when
dropping objects\n"));
Can this be changed to dump objects and data based on the filter
expressions from the filter file.

I am sorry, I don't understand. This should work for data from specified by
filter without any modification.

attached updated patch

Regards

Pavel

Show quoted text

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

Attachments:

pg_dump-filter-20200713-2.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200713-2.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7a37fd8045..2f2bfb4dbf 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,99 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        This option ensure reading object's filters from specified file.
+        If you use "-" as filename, then stdin is used as source. This file
+        has to have following line format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+        Only one object name can be specified per one line:
+<programlisting>
++t mytable1
++t mytable2
++f some_foreign_table
+-d mytable3
+</programlisting>
+        With this file the dump ensures dump table <literal>mytable1</literal>,
+        <literal>mytable2</literal>. The data of foreign table
+        <literal>some_foreign_table</literal> will be dumped too. And the data
+        of <literal>mytable3</literal> will not be dumped.
+       </para>
+
+       <para>
+        The first char <literal>+</literal> or <literal>-</literal> specifies
+        if object name will be used as include or exclude filter.
+       </para>
+
+       <para>
+        The second char
+        <literal>t</literal>,
+        <literal>n</literal>,
+        <literal>f</literal>,
+        <literal>d</literal>
+        specifies a object type.
+
+        <variablelist>
+         <varlistentry>
+          <term><literal>t</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--table</option>. In exclusive form (<literal>-</literal>)
+            it is same like <option>--exclude-table</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>n</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--schema</option>. In exclusive form (<literal>-</literal>)
+            it is same like <option>--exclude-schema</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>f</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--include-foreign-data</option>. The exclusive form
+            (<literal>-</literal>) is not allowed.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>d</literal></term>
+          <listitem>
+           <para>
+            The inclusive form (<literal>+</literal>) is not allowed.
+            In exclusive form (<literal>-</literal>) it is same like
+            <option>--exclude-table-data</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+        </variablelist>
+       </para>
+
+       <para>
+        The option <option>--filter</option> can be used together with options
+        <option>--table</option>, <option>--exclude-table</option>,
+        <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--include-foreign-data</option> and
+        <option>--exclude-table-data</option>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e758b5c50a..fd6b7a174a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -364,6 +365,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +605,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1028,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object name filter expressions from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18597,3 +18604,211 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+#define		FILTER_INITIAL_LINE_SIZE		1024
+#define		PG_GETLINE_EXTEND_LINE_SIZE		1024
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += PG_GETLINE_EXTEND_LINE_SIZE;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	char	   *line;
+	ssize_t		chars;
+	size_t		line_size = FILTER_INITIAL_LINE_SIZE;
+	int			lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	line = pg_malloc(line_size);
+
+	while ((chars = pg_getline(&line, &line_size, fp)) != -1)
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+
+		lineno += 1;
+
+		if (line[chars - 1] == '\n')
+			line[chars - 1] = '\0';
+
+		/* ignore empty rows */
+		if (*line == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*line == '#')
+			continue;
+
+		if (chars < 2)
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "too short line",
+									   line,
+									   lineno);
+
+		if (line[0] == '+')
+			is_include = true;
+		else if (line[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   line,
+									   lineno);
+
+		objecttype = line[1];
+		objectname = &line[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   line,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   line,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   line,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   line,
+									   lineno);
+	}
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	pg_free(line);
+}
#26Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#25)
Re: proposal: possibility to read dumped table's name from file

On 13 Jul 2020, at 10:20, Pavel Stehule <pavel.stehule@gmail.com> wrote:

attached updated patch

Sorry for jumping in late, but thinking about this extension to pg_dump:
doesn't it make more sense to use an existing file format like JSON for this,
given that virtually all devops/cd/etc tooling know about JSON already?

Considering its application and the problem space, I'd expect users to generate
this file rather than handcraft it with 10 rows of content, and standard file
formats help there. Creative users could even use the database itself to
easily manage its content and generate the file (which isn't limited to JSON of
course, but it would be easier). Also, we now have backup manifests in JSON
which IMO sets a bit of a precedent, even though thats a separate thing.

At the very least it seems limiting to not include a file format version
identifier since we'd otherwise risk running into backwards compat issues
should we want to expand on this in the future.

cheers ./daniel

#27Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#26)
Re: proposal: possibility to read dumped table's name from file

po 13. 7. 2020 v 12:04 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 13 Jul 2020, at 10:20, Pavel Stehule <pavel.stehule@gmail.com> wrote:

attached updated patch

Sorry for jumping in late, but thinking about this extension to pg_dump:
doesn't it make more sense to use an existing file format like JSON for
this,
given that virtually all devops/cd/etc tooling know about JSON already?

Considering its application and the problem space, I'd expect users to
generate
this file rather than handcraft it with 10 rows of content, and standard
file
formats help there. Creative users could even use the database itself to
easily manage its content and generate the file (which isn't limited to
JSON of
course, but it would be easier). Also, we now have backup manifests in
JSON
which IMO sets a bit of a precedent, even though thats a separate thing.

At the very least it seems limiting to not include a file format version
identifier since we'd otherwise risk running into backwards compat issues
should we want to expand on this in the future.

I like JSON format. But why here? For this purpose the JSON is over
engineered. This input file has no nested structure - it is just a stream
of lines.

I don't think so introducing JSON here can be a good idea. For this feature
typical usage can be used in pipe, and the most simple format (what is
possible) is ideal.

It is a really different case than pg_dump manifest file - in this case, in
this case pg_dump is consument.

Regards

Pavel

Show quoted text

cheers ./daniel

#28vignesh C
vignesh21@gmail.com
In reply to: Pavel Stehule (#25)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jul 13, 2020 at 1:51 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

Can this be changed to dump objects and data based on the filter
expressions from the filter file.

I am sorry, I don't understand. This should work for data from specified by filter without any modification.

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

#29Justin Pryzby
pryzby@telsasoft.com
In reply to: Daniel Gustafsson (#26)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jul 13, 2020 at 12:04:09PM +0200, Daniel Gustafsson wrote:

Sorry for jumping in late, but thinking about this extension to pg_dump:
doesn't it make more sense to use an existing file format like JSON for this,
given that virtually all devops/cd/etc tooling know about JSON already?

Considering its application and the problem space, I'd expect users to generate
this file rather than handcraft it with 10 rows of content, and standard file
formats help there.

I mentioned having tested this patch as we would use it. But it's likely I
*wouldn't* use it if the format was something which required added complexity
to pipe in from an existing shell script.

At the very least it seems limiting to not include a file format version
identifier since we'd otherwise risk running into backwards compat issues
should we want to expand on this in the future.

Maybe .. I'm not sure. The patch would of course be extended to handle
additional include/exclude options. Is there any other future behavior we
might reasonably anticipate ?

If at some point we wanted to support another file format, maybe it would look
like: --format=v2:filters.txt (or maybe the old one would be v1:filters.txt)

--
Justin

#30Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#27)
Re: proposal: possibility to read dumped table's name from file

On 13 Jul 2020, at 13:02, Pavel Stehule <pavel.stehule@gmail.com> wrote:

I like JSON format. But why here? For this purpose the JSON is over engineered.

I respectfully disagree, JSON is a commonly used and known format in systems
administration and most importantly: we already have code to parse it in the
frontend.

This input file has no nested structure - it is just a stream of lines.

Well, it has a set of object types which in turn have objects. There is more
structure than meets the eye.

Also, the current patch allows arbitrary whitespace before object names, but no
whitespace before comments etc. Using something where the rules of parsing are
known is rarely a bad thing.

I don't think so introducing JSON here can be a good idea.

Quite possibly it isn't, but not discussing options seems like a worse idea so
I wanted to bring it up.

It is a really different case than pg_dump manifest file - in this case, in this case pg_dump is consument.

Right, as I said these are two different, while tangentially related, things.

cheers ./daniel

#31Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#30)
Re: proposal: possibility to read dumped table's name from file

po 13. 7. 2020 v 16:57 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 13 Jul 2020, at 13:02, Pavel Stehule <pavel.stehule@gmail.com> wrote:

I like JSON format. But why here? For this purpose the JSON is over

engineered.

I respectfully disagree, JSON is a commonly used and known format in
systems
administration and most importantly: we already have code to parse it in
the
frontend.

I disagree with the idea so if we have a client side JSON parser we have
to use it everywhere.
For this case, parsing JSON means more code, not less. I checked the
parse_manifest.c. More
the JSON API is DOM type. For this purpose the SAX type is better. But
still, things should be simple as possible.
There is not any necessity to use it.

JSON is good for a lot of purposes, and can be good if the document uses
more lexer types, numeric, ... But nothing is used there

This input file has no nested structure - it is just a stream of lines.

Well, it has a set of object types which in turn have objects. There is
more
structure than meets the eye.

Also, the current patch allows arbitrary whitespace before object names,
but no
whitespace before comments etc. Using something where the rules of
parsing are
known is rarely a bad thing.

if I know - JSON hasn't comments at all.

I don't think so introducing JSON here can be a good idea.

Quite possibly it isn't, but not discussing options seems like a worse
idea so
I wanted to bring it up.

It is a really different case than pg_dump manifest file - in this case,

in this case pg_dump is consument.

Right, as I said these are two different, while tangentially related,
things.

Backup manifest format has no trivial complexity - and using JSON has
sense. Input filter file is a trivial - +/- list of strings (and it will be
everytime).

In this case I don't see any benefits from JSON - on both sides (producent,
consuments). It is harder (little bit) to parse it, it is harder (little
bit) to generate it.

Regards

Pavel

Show quoted text

cheers ./daniel

#32Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#24)
2 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jul 13, 2020 at 08:15:42AM +0200, Pavel Stehule wrote:

Do you want to add any more documentation ?

done

Thanks - I think the documentation was maybe excessive. See attached.

--
Justin

Attachments:

0001-proposal-possibility-to-read-dumped-table-s-name-fro.patchtext/x-diff; charset=us-asciiDownload
From b6ceedcd7f4395fac822059229cb475aa2805c1e Mon Sep 17 00:00:00 2001
From: Pavel Stehule <pavel.stehule@gmail.com>
Date: Mon, 13 Jul 2020 10:20:42 +0200
Subject: [PATCH 1/2] proposal: possibility to read dumped table's name from
 file

---
 doc/src/sgml/ref/pg_dump.sgml |  93 +++++++++++++++
 src/bin/pg_dump/pg_dump.c     | 215 ++++++++++++++++++++++++++++++++++
 2 files changed, 308 insertions(+)

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7a37fd8045..2f2bfb4dbf 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,99 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        This option ensure reading object's filters from specified file.
+        If you use "-" as filename, then stdin is used as source. This file
+        has to have following line format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+        Only one object name can be specified per one line:
+<programlisting>
++t mytable1
++t mytable2
++f some_foreign_table
+-d mytable3
+</programlisting>
+        With this file the dump ensures dump table <literal>mytable1</literal>,
+        <literal>mytable2</literal>. The data of foreign table
+        <literal>some_foreign_table</literal> will be dumped too. And the data
+        of <literal>mytable3</literal> will not be dumped.
+       </para>
+
+       <para>
+        The first char <literal>+</literal> or <literal>-</literal> specifies
+        if object name will be used as include or exclude filter.
+       </para>
+
+       <para>
+        The second char
+        <literal>t</literal>,
+        <literal>n</literal>,
+        <literal>f</literal>,
+        <literal>d</literal>
+        specifies a object type.
+
+        <variablelist>
+         <varlistentry>
+          <term><literal>t</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--table</option>. In exclusive form (<literal>-</literal>)
+            it is same like <option>--exclude-table</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>n</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--schema</option>. In exclusive form (<literal>-</literal>)
+            it is same like <option>--exclude-schema</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>f</literal></term>
+          <listitem>
+           <para>
+            In inclusive form (<literal>+</literal>) it does same work like
+            <option>--include-foreign-data</option>. The exclusive form
+            (<literal>-</literal>) is not allowed.
+           </para>
+          </listitem>
+         </varlistentry>
+
+         <varlistentry>
+          <term><literal>d</literal></term>
+          <listitem>
+           <para>
+            The inclusive form (<literal>+</literal>) is not allowed.
+            In exclusive form (<literal>-</literal>) it is same like
+            <option>--exclude-table-data</option>.
+           </para>
+          </listitem>
+         </varlistentry>
+        </variablelist>
+       </para>
+
+       <para>
+        The option <option>--filter</option> can be used together with options
+        <option>--table</option>, <option>--exclude-table</option>,
+        <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--include-foreign-data</option> and
+        <option>--exclude-table-data</option>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e758b5c50a..fd6b7a174a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -364,6 +365,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +605,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1028,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            read object name filter expressions from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18597,3 +18604,211 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+#define		FILTER_INITIAL_LINE_SIZE		1024
+#define		PG_GETLINE_EXTEND_LINE_SIZE		1024
+
+/*
+ * getline is originaly GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += PG_GETLINE_EXTEND_LINE_SIZE;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	char	   *line;
+	ssize_t		chars;
+	size_t		line_size = FILTER_INITIAL_LINE_SIZE;
+	int			lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	line = pg_malloc(line_size);
+
+	while ((chars = pg_getline(&line, &line_size, fp)) != -1)
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+
+		lineno += 1;
+
+		if (line[chars - 1] == '\n')
+			line[chars - 1] = '\0';
+
+		/* ignore empty rows */
+		if (*line == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*line == '#')
+			continue;
+
+		if (chars < 2)
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "too short line",
+									   line,
+									   lineno);
+
+		if (line[0] == '+')
+			is_include = true;
+		else if (line[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   line,
+									   lineno);
+
+		objecttype = line[1];
+		objectname = &line[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   line,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   line,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   line,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   line,
+									   lineno);
+	}
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	pg_free(line);
+}
-- 
2.17.0

0002-fixen.patchtext/x-diff; charset=us-asciiDownload
From 56c66987d1b645a14932f5051ecbb983956e8b31 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 13 Jul 2020 12:57:34 -0500
Subject: [PATCH 2/2] fixen

---
 doc/src/sgml/ref/pg_dump.sgml | 107 +++++++++-------------------------
 src/bin/pg_dump/pg_dump.c     |   4 +-
 2 files changed, 31 insertions(+), 80 deletions(-)

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 2f2bfb4dbf..c9502dca13 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -759,91 +759,42 @@ PostgreSQL documentation
       <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
       <listitem>
        <para>
-        This option ensure reading object's filters from specified file.
-        If you use "-" as filename, then stdin is used as source. This file
-        has to have following line format:
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
 <synopsis>
 (+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
 </synopsis>
-        Only one object name can be specified per one line:
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign table),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign table
+        <literal>some_foreign_table</literal>, but exclude data
+        from table <literal>mytable2</literal>.
 <programlisting>
 +t mytable1
-+t mytable2
 +f some_foreign_table
--d mytable3
+-d mytable2
 </programlisting>
-        With this file the dump ensures dump table <literal>mytable1</literal>,
-        <literal>mytable2</literal>. The data of foreign table
-        <literal>some_foreign_table</literal> will be dumped too. And the data
-        of <literal>mytable3</literal> will not be dumped.
-       </para>
-
-       <para>
-        The first char <literal>+</literal> or <literal>-</literal> specifies
-        if object name will be used as include or exclude filter.
-       </para>
-
-       <para>
-        The second char
-        <literal>t</literal>,
-        <literal>n</literal>,
-        <literal>f</literal>,
-        <literal>d</literal>
-        specifies a object type.
-
-        <variablelist>
-         <varlistentry>
-          <term><literal>t</literal></term>
-          <listitem>
-           <para>
-            In inclusive form (<literal>+</literal>) it does same work like
-            <option>--table</option>. In exclusive form (<literal>-</literal>)
-            it is same like <option>--exclude-table</option>.
-           </para>
-          </listitem>
-         </varlistentry>
-
-         <varlistentry>
-          <term><literal>n</literal></term>
-          <listitem>
-           <para>
-            In inclusive form (<literal>+</literal>) it does same work like
-            <option>--schema</option>. In exclusive form (<literal>-</literal>)
-            it is same like <option>--exclude-schema</option>.
-           </para>
-          </listitem>
-         </varlistentry>
-
-         <varlistentry>
-          <term><literal>f</literal></term>
-          <listitem>
-           <para>
-            In inclusive form (<literal>+</literal>) it does same work like
-            <option>--include-foreign-data</option>. The exclusive form
-            (<literal>-</literal>) is not allowed.
-           </para>
-          </listitem>
-         </varlistentry>
-
-         <varlistentry>
-          <term><literal>d</literal></term>
-          <listitem>
-           <para>
-            The inclusive form (<literal>+</literal>) is not allowed.
-            In exclusive form (<literal>-</literal>) it is same like
-            <option>--exclude-table-data</option>.
-           </para>
-          </listitem>
-         </varlistentry>
-        </variablelist>
-       </para>
-
-       <para>
-        The option <option>--filter</option> can be used together with options
-        <option>--table</option>, <option>--exclude-table</option>,
-        <option>--schema</option>, <option>--exclude-schema</option>,
-        <option>--include-foreign-data</option> and
-        <option>--exclude-table-data</option>.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
        </para>
       </listitem>
      </varlistentry>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index fd6b7a174a..9f3da36cd5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -18609,7 +18609,7 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 #define		PG_GETLINE_EXTEND_LINE_SIZE		1024
 
 /*
- * getline is originaly GNU function, and should not be everywhere still.
+ * getline is originally GNU function, and should not be everywhere still.
  * Use own reduced implementation.
  */
 static size_t
@@ -18719,7 +18719,7 @@ read_patterns_from_file(char *filename, DumpOptions *dopt)
 		if (chars < 2)
 			exit_invalid_filter_format(fp,
 									   filename,
-									   "too short line",
+									   "line too short",
 									   line,
 									   lineno);
 
-- 
2.17.0

#33Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#32)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

po 13. 7. 2020 v 20:00 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Mon, Jul 13, 2020 at 08:15:42AM +0200, Pavel Stehule wrote:

Do you want to add any more documentation ?

done

Thanks - I think the documentation was maybe excessive. See attached.

I merged your patch - thank you

new patch with doc changes and text of help change requested by Vignesh
attached

Regards

Pavel

Show quoted text

--
Justin

Attachments:

pg_dump-filter-20200714.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200714.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7a37fd8045..c9502dca13 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,50 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign table),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign table
+        <literal>some_foreign_table</literal>, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_table
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e758b5c50a..778ae2d0fc 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -364,6 +365,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +605,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1028,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18597,3 +18605,211 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+#define		FILTER_INITIAL_LINE_SIZE		1024
+#define		PG_GETLINE_EXTEND_LINE_SIZE		1024
+
+/*
+ * getline is originally GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += PG_GETLINE_EXTEND_LINE_SIZE;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	char	   *line;
+	ssize_t		chars;
+	size_t		line_size = FILTER_INITIAL_LINE_SIZE;
+	int			lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	line = pg_malloc(line_size);
+
+	while ((chars = pg_getline(&line, &line_size, fp)) != -1)
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+
+		lineno += 1;
+
+		if (line[chars - 1] == '\n')
+			line[chars - 1] = '\0';
+
+		/* ignore empty rows */
+		if (*line == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*line == '#')
+			continue;
+
+		if (chars < 2)
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   line,
+									   lineno);
+
+		if (line[0] == '+')
+			is_include = true;
+		else if (line[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   line,
+									   lineno);
+
+		objecttype = line[1];
+		objectname = &line[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   line,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   line,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   line,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   line,
+									   lineno);
+	}
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	pg_free(line);
+}
#34Pavel Stehule
pavel.stehule@gmail.com
In reply to: vignesh C (#28)
Re: proposal: possibility to read dumped table's name from file

po 13. 7. 2020 v 15:33 odesílatel vignesh C <vignesh21@gmail.com> napsal:

On Mon, Jul 13, 2020 at 1:51 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Can this be changed to dump objects and data based on the filter
expressions from the filter file.

I am sorry, I don't understand. This should work for data from specified

by filter without any modification.

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

done in today patch

Pavel

Show quoted text

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

#35vignesh C
vignesh21@gmail.com
In reply to: Pavel Stehule (#34)
Re: proposal: possibility to read dumped table's name from file

On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

done in today patch

Thanks for fixing the  comments.
Few comments:
+ /* use "-" as symbol for stdin */
+ if (strcmp(filename, "-") != 0)
+ {
+ fp = fopen(filename, "r");
+ if (!fp)
+ fatal("could not open the input file \"%s\": %m",
+   filename);
+ }
+ else
+ fp = stdin;

We could use STDIN itself instead of -, it will be a more easier
option to understand.

+ /* when first char is hash, ignore whole line */
+ if (*line == '#')
+ continue;

If line starts with # we ignore that line, I feel this should be
included in the documentation.

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

#36Justin Pryzby
pryzby@telsasoft.com
In reply to: vignesh C (#35)
Re: proposal: possibility to read dumped table's name from file

On Sat, Jul 25, 2020 at 06:56:31PM +0530, vignesh C wrote:

On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

done in today patch

Thanks for fixing the  comments.
Few comments:
+ /* use "-" as symbol for stdin */
+ if (strcmp(filename, "-") != 0)
+ {
+ fp = fopen(filename, "r");
+ if (!fp)
+ fatal("could not open the input file \"%s\": %m",
+   filename);
+ }
+ else
+ fp = stdin;

We could use STDIN itself instead of -, it will be a more easier
option to understand.

I think "-" is used widely for commandline tools, and STDIN is not (even though
it's commonly used by programmers). For example, since last year, pg_restore
-f - means stdout.

--
Justin

#37Pavel Stehule
pavel.stehule@gmail.com
In reply to: vignesh C (#35)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

so 25. 7. 2020 v 15:26 odesílatel vignesh C <vignesh21@gmail.com> napsal:

On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

done in today patch

Thanks for fixing the  comments.
Few comments:
+ /* use "-" as symbol for stdin */
+ if (strcmp(filename, "-") != 0)
+ {
+ fp = fopen(filename, "r");
+ if (!fp)
+ fatal("could not open the input file \"%s\": %m",
+   filename);
+ }
+ else
+ fp = stdin;

We could use STDIN itself instead of -, it will be a more easier
option to understand.

+ /* when first char is hash, ignore whole line */
+ if (*line == '#')
+ continue;

If line starts with # we ignore that line, I feel this should be
included in the documentation.

Good note - I wrote sentence to doc

+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes.
+       </para>
+
Show quoted text

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com

Attachments:

pg_dump-filter-20200727.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200727.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7a37fd8045..0e0fbc90db 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign table),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign table
+        <literal>some_foreign_table</literal>, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_table
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 94459b3539..8df1f91cb6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -290,6 +290,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -364,6 +365,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +605,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1028,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18434,3 +18442,211 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+#define		FILTER_INITIAL_LINE_SIZE		1024
+#define		PG_GETLINE_EXTEND_LINE_SIZE		1024
+
+/*
+ * getline is originally GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{
+	size_t		total_chars = 0;
+
+	while (!feof(fp) && !ferror(fp))
+	{
+		char	   *str;
+		size_t		chars;
+
+		str = fgets(*lineptr + total_chars,
+					*n - total_chars,
+					fp);
+
+		if (ferror(fp))
+			return -1;
+
+		if (str)
+		{
+			chars = strlen(str);
+			total_chars += chars;
+
+			if (chars > 0 && str[chars - 1] == '\n')
+				return total_chars;
+
+			/* check, if there is good enough space for next content */
+			if (*n - total_chars < 2)
+			{
+				*n += PG_GETLINE_EXTEND_LINE_SIZE;
+				*lineptr = pg_realloc(*lineptr, *n);
+			}
+		}
+		else
+			break;
+	}
+
+	if (ferror(fp))
+		return -1;
+
+	return total_chars > 0 ? total_chars : -1;
+}
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	char	   *line;
+	ssize_t		chars;
+	size_t		line_size = FILTER_INITIAL_LINE_SIZE;
+	int			lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	line = pg_malloc(line_size);
+
+	while ((chars = pg_getline(&line, &line_size, fp)) != -1)
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+
+		lineno += 1;
+
+		if (line[chars - 1] == '\n')
+			line[chars - 1] = '\0';
+
+		/* ignore empty rows */
+		if (*line == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*line == '#')
+			continue;
+
+		if (chars < 2)
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   line,
+									   lineno);
+
+		if (line[0] == '+')
+			is_include = true;
+		else if (line[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   line,
+									   lineno);
+
+		objecttype = line[1];
+		objectname = &line[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   line,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   line,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   line,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   line,
+									   lineno);
+	}
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	pg_free(line);
+}
#38Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#36)
Re: proposal: possibility to read dumped table's name from file

ne 26. 7. 2020 v 21:10 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Sat, Jul 25, 2020 at 06:56:31PM +0530, vignesh C wrote:

On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule <pavel.stehule@gmail.com>

wrote:

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

done in today patch

Thanks for fixing the  comments.
Few comments:
+ /* use "-" as symbol for stdin */
+ if (strcmp(filename, "-") != 0)
+ {
+ fp = fopen(filename, "r");
+ if (!fp)
+ fatal("could not open the input file \"%s\": %m",
+   filename);
+ }
+ else
+ fp = stdin;

We could use STDIN itself instead of -, it will be a more easier
option to understand.

I think "-" is used widely for commandline tools, and STDIN is not (even
though
it's commonly used by programmers). For example, since last year,
pg_restore
-f - means stdout.

yes, STDIN is used by programming languages, but it is not usual in command
line tools. And because it was used by pg_restore, then we should not use
new inconsistency.

Regards

Pavel

Show quoted text

--
Justin

#39Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#37)
Re: proposal: possibility to read dumped table's name from file

On Mon, Jul 27, 2020 at 07:25:54AM +0200, Pavel Stehule wrote:

so 25. 7. 2020 v 15:26 odes�latel vignesh C <vignesh21@gmail.com> napsal:

On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

I meant can this:
printf(_(" --filter=FILENAME read object name filter
expressions from file\n"));
be changed to:
printf(_(" --filter=FILENAME dump objects and data based
on the filter expressions from the filter file\n"));

done in today patch

This looks good to my eyes - marked RFC.

--
Justin

#40Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#18)
Re: proposal: possibility to read dumped table's name from file

On Sun, Jul 05, 2020 at 10:08:09PM +0200, Pavel Stehule wrote:

st 1. 7. 2020 v 23:24 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote:

st 10. 6. 2020 v 0:30 odes�latel Justin Pryzby <pryzby@telsasoft.com>> napsal:

Also, your getline is dynamically re-allocating lines of arbitrary length.
Possibly that's not needed. We'll typically read "+t schema.relname",
which is
132 chars. Maybe it's sufficient to do
char buf[1024];
fgets(buf);
if strchr(buf, '\n') == NULL: error();
ret = pstrdup(buf);

63 bytes is max effective identifier size, but it is not max size of
identifiers. It is very probably so buff with 1024 bytes will be enough for
all, but I do not want to increase any new magic limit. More when dynamic
implementation is not too hard.

Maybe you'd want to use a StrInfo like recent patches (8f8154a50).

Table name can be very long - sometimes the data names (table names) can be
stored in external storages with full length and should not be practical to
require truncating in filter file.

For this case it is very effective, because a resized (increased) buffer is
used for following rows, so realloc should not be often. So when I have to
choose between two implementations with similar complexity, I prefer more
dynamic code without hardcoded limits. This dynamic hasn't any overhead.

--
Justin

#41Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Pavel Stehule (#37)
Re: proposal: possibility to read dumped table's name from file

On 2020-Jul-27, Pavel Stehule wrote:

+/*
+ * getline is originally GNU function, and should not be everywhere still.
+ * Use own reduced implementation.
+ */
+static size_t
+pg_getline(char **lineptr, size_t *n, FILE *fp)
+{

So, Tom added a coding pattern for doing this in commit 8f8154a503c7,
which is ostensibly also to be used in pg_regress [1]/messages/by-id/m_1NfbowTqSJnrC6rq1a9cQK7E-CHQE7B6Kz9w6fNH-OiV-4mcsdMw7UP2oA2_6dZmXvAMjbSPZjW9U7FD2R52D3d9DtaJxcBprsqJqZNBc=@protonmail.com -- maybe it'd be
useful to have this in src/common?

[1]: /messages/by-id/m_1NfbowTqSJnrC6rq1a9cQK7E-CHQE7B6Kz9w6fNH-OiV-4mcsdMw7UP2oA2_6dZmXvAMjbSPZjW9U7FD2R52D3d9DtaJxcBprsqJqZNBc=@protonmail.com

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#42Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#41)
Re: proposal: possibility to read dumped table's name from file

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

So, Tom added a coding pattern for doing this in commit 8f8154a503c7,
which is ostensibly also to be used in pg_regress [1] -- maybe it'd be
useful to have this in src/common?

Done, see pg_get_line() added by 67a472d71.

regards, tom lane

#43Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#42)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

pá 4. 9. 2020 v 2:15 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

So, Tom added a coding pattern for doing this in commit 8f8154a503c7,
which is ostensibly also to be used in pg_regress [1] -- maybe it'd be
useful to have this in src/common?

Done, see pg_get_line() added by 67a472d71.

Here is updated patch for pg_dump

Regards

Pavel

Show quoted text

regards, tom lane

Attachments:

pg_dump_filter-20200904.patchtext/x-patch; charset=US-ASCII; name=pg_dump_filter-20200904.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0b2e2de87b..48dd75bfb1 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign table),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign table
+        <literal>some_foreign_table</literal>, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_table
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d3ca54e4dc..1d3c97ad97 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -53,6 +53,7 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
@@ -291,6 +292,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -365,6 +367,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -604,6 +607,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1023,6 +1030,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18473,3 +18482,157 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	char	   *line;
+	int			lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	while ((line = pg_get_line(fp)))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(line);
+
+		/* ignore empty rows */
+		if (*line == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*line == '#')
+			continue;
+
+		if (line[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   line,
+									   lineno);
+
+		if (line[0] == '+')
+			is_include = true;
+		else if (line[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   line,
+									   lineno);
+
+		objecttype = line[1];
+		objectname = &line[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   line,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   line,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   line,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   line,
+									   lineno);
+
+		pfree(line);
+	}
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
#44Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#43)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

pá 4. 9. 2020 v 5:21 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

pá 4. 9. 2020 v 2:15 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

So, Tom added a coding pattern for doing this in commit 8f8154a503c7,
which is ostensibly also to be used in pg_regress [1] -- maybe it'd be
useful to have this in src/common?

Done, see pg_get_line() added by 67a472d71.

Here is updated patch for pg_dump

another update based on pg_get_line_append function

Regards

Pavel

Show quoted text

Regards

Pavel

regards, tom lane

Attachments:

pg_dump-filter-20200907.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200907.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0b2e2de87b..48dd75bfb1 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign table),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign table
+        <literal>some_foreign_table</literal>, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_table
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 784bceaec3..6349be55e9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -53,9 +53,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -291,6 +293,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -365,6 +368,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -604,6 +608,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1023,6 +1031,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18477,3 +18487,162 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_append(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty rows */
+		if (*str == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+
+		resetStringInfo(&line);
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
#45Surafel Temesgen
surafel3000@gmail.com
In reply to: Pavel Stehule (#43)
Re: proposal: possibility to read dumped table's name from file

Hi Pavel

On Fri, Sep 4, 2020 at 6:22 AM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Here is updated patch for pg_dump

pg_dumpall also has –exclude-database=pattern and –no-comments option
doesn't that qualify it to benefits from this feature? And please add a
test case for this option

regards

Surafel

#46Pavel Stehule
pavel.stehule@gmail.com
In reply to: Surafel Temesgen (#45)
Re: proposal: possibility to read dumped table's name from file

Hi

po 7. 9. 2020 v 14:14 odesílatel Surafel Temesgen <surafel3000@gmail.com>
napsal:

Hi Pavel

On Fri, Sep 4, 2020 at 6:22 AM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Here is updated patch for pg_dump

pg_dumpall also has –exclude-database=pattern and –no-comments option
doesn't that qualify it to benefits from this feature? And please add a
test case for this option

This patch is related to pg_dump (in this moment), so pg_dumpall options
are out of scope.

I am not sure if pg_dumpall needs this functionality - maybe, but I can use
bash or some similar for implementation of this feature. There is no
requirement to do it all necessary work under one transaction, one snapshot.

For pg_dump can be used different format, because it uses different
granularity. Some like "{+/-} dbname"

"--no-comments" is a global parameter without arguments. I don't understand
how this parameter can be related to this feature?

I am working on regress tests.

Regards

Pavel

Show quoted text

regards

Surafel

#47Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#46)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

pá 11. 9. 2020 v 10:50 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

po 7. 9. 2020 v 14:14 odesílatel Surafel Temesgen <surafel3000@gmail.com>
napsal:

Hi Pavel

On Fri, Sep 4, 2020 at 6:22 AM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Here is updated patch for pg_dump

pg_dumpall also has –exclude-database=pattern and –no-comments option
doesn't that qualify it to benefits from this feature? And please add a
test case for this option

This patch is related to pg_dump (in this moment), so pg_dumpall options
are out of scope.

I am not sure if pg_dumpall needs this functionality - maybe, but I can
use bash or some similar for implementation of this feature. There is no
requirement to do it all necessary work under one transaction, one snapshot.

For pg_dump can be used different format, because it uses different
granularity. Some like "{+/-} dbname"

"--no-comments" is a global parameter without arguments. I don't
understand how this parameter can be related to this feature?

I am working on regress tests.

There is a updated version with regress tests

Regards

Pavel

Show quoted text

Regards

Pavel

regards

Surafel

Attachments:

pg_dump-filter-20200912.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200912.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0b2e2de87b..c4f818f305 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -755,6 +755,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign server),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_server
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 784bceaec3..94a8b2e187 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -53,9 +53,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -291,6 +293,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -365,6 +368,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -604,6 +608,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1023,6 +1031,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18477,3 +18487,164 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_append(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty rows */
+		if (*str == '\0')
+			goto resetline;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			goto resetline;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+
+resetline:
+
+		resetStringInfo(&line);
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
new file mode 100644
index 0000000000..957442c5e4
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -0,0 +1,129 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 22;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+t table_one\n";
+print $inputfile "+t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "-d table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-n public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+f doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+d sometable";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/include filter is not supported for this type of object/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-f someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not supported for this type of object/,
+	"broken format check");
#48Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#47)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

rebase + minor change - using pg_get_line_buf instead pg_get_line_append

Regards

Pavel

Attachments:

pg_dump-filter-20200924.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20200924.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index e09ed0a4c3..9d7ebaf922 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -757,6 +757,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign server),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_server
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f021bb72f4..a7824c5bf6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -53,9 +53,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -290,6 +292,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -364,6 +367,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -603,6 +607,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1022,6 +1030,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18470,3 +18480,160 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty rows */
+		if (*str == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
new file mode 100644
index 0000000000..957442c5e4
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -0,0 +1,129 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 22;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+t table_one\n";
+print $inputfile "+t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "-d table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-n public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+f doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+d sometable";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/include filter is not supported for this type of object/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-f someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not supported for this type of object/,
+	"broken format check");
#49Stephen Frost
sfrost@snowman.net
In reply to: Pavel Stehule (#48)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Pavel Stehule (pavel.stehule@gmail.com) wrote:

rebase + minor change - using pg_get_line_buf instead pg_get_line_append

I started looking at this and went back through the thread and while I
tend to agree that JSON may not be a good choice for this, it's not the
only possible alternative. There is no doubt that pg_dump is already a
sophisticated data export tool, and likely to continue to gain new
features, such that having a configuration file for it would be very
handy, but this clearly isn't really going in a direction that would
allow for that.

Perhaps this feature could co-exist with a full blown configuration for
pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a newline
in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address these
issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters in
the future.

Unlike for the pg_basebackup manifest, which we generate and read
entirely programatically, a config file for pg_dump would almost
certainly be updated manually (or, at least, parts of it would be and
perhaps other parts generated), which means it'd really be ideal to have
a proper way to support comments in it (something that the proposed
format also doesn't really get right- # must be the *first* character,
and you can only have whole-line comments..?), avoid extra unneeded
punctuation (or, at times, allow it- such as trailing commas in lists),
cleanly handle multi-line strings (consider the oft discussed idea
around having pg_dump support a WHERE clause for exporting data from
tables...), etc.

Overall, -1 from me on this approach. Maybe it could be fixed up to
handle all the different names of objects that we support today
(something which, imv, is really a clear requirement for this feature to
be committed), but I suspect you'd end up half-way to yet another
configuration format when we could be working to support something like
TOML or maybe YAML... but if you want my 2c, TOML seems closer to what
we do for postgresql.conf and getting that over to something that's
standardized, while a crazy long shot, is a general nice idea, imv.

Thanks,

Stephen

#50Pavel Stehule
pavel.stehule@gmail.com
In reply to: Stephen Frost (#49)
Re: proposal: possibility to read dumped table's name from file

Hi

út 10. 11. 2020 v 21:09 odesílatel Stephen Frost <sfrost@snowman.net>
napsal:

Greetings,

* Pavel Stehule (pavel.stehule@gmail.com) wrote:

rebase + minor change - using pg_get_line_buf instead pg_get_line_append

I started looking at this and went back through the thread and while I
tend to agree that JSON may not be a good choice for this, it's not the
only possible alternative. There is no doubt that pg_dump is already a
sophisticated data export tool, and likely to continue to gain new
features, such that having a configuration file for it would be very
handy, but this clearly isn't really going in a direction that would
allow for that.

Perhaps this feature could co-exist with a full blown configuration for
pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a newline
in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address these
issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters in
the future.

This is the correct argument - I will check a possibility to use strange
names, but there is the same possibility and functionality like we allow
from the command line. So you can use double quoted names. I'll check it.

Unlike for the pg_basebackup manifest, which we generate and read
entirely programatically, a config file for pg_dump would almost
certainly be updated manually (or, at least, parts of it would be and
perhaps other parts generated), which means it'd really be ideal to have
a proper way to support comments in it (something that the proposed
format also doesn't really get right- # must be the *first* character,
and you can only have whole-line comments..?), avoid extra unneeded
punctuation (or, at times, allow it- such as trailing commas in lists),
cleanly handle multi-line strings (consider the oft discussed idea
around having pg_dump support a WHERE clause for exporting data from
tables...), etc.

I think the proposed feature is very far to be the config file for pg_dump
(it implements a option "--filter"). This is not the target. It is not
designed for this. This is just an alternative for options like -t, -T, ...
and I am sure so nobody will generate this file manually. Main target of
this patch is eliminating problems with the max length of the command line.
So it is really not designed to be the config file for pg_dump.

Overall, -1 from me on this approach. Maybe it could be fixed up to
handle all the different names of objects that we support today
(something which, imv, is really a clear requirement for this feature to
be committed), but I suspect you'd end up half-way to yet another
configuration format when we could be working to support something like
TOML or maybe YAML... but if you want my 2c, TOML seems closer to what
we do for postgresql.conf and getting that over to something that's
standardized, while a crazy long shot, is a general nice idea, imv.

I have nothing against TOML, but I don't see a sense of usage in this
patch. This patch doesn't implement a config file for pg_dump, and I don't
see any sense or benefits of it. The TOML is designed for different
purposes. TOML is good for manual creating, but it is not this case.
Typical usage of this patch is some like, and TOML syntax (or JSON) is not
good for this.

psql -c "select '+t' || quote_ident(relname) from pg_class where relname
..." | pg_dump --filter=/dev/stdin

I can imagine some benefits of saved configure files for postgres
applications - but it should be designed generally and implemented
generally. Probably you would use one for pg_dump, psql, pg_restore, ....
But it is a different feature with different usage. This patch doesn't
implement option "--config", it implements option "--filter".

Regards

Pavel

Show quoted text

Thanks,

Stephen

#51Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#50)
Re: proposal: possibility to read dumped table's name from file

Hi

Perhaps this feature could co-exist with a full blown configuration for

pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a newline
in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address these
issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters in
the future.

This is the correct argument - I will check a possibility to use strange
names, but there is the same possibility and functionality like we allow
from the command line. So you can use double quoted names. I'll check it.

I checked

echo "+t \"bad Name\"" | /usr/local/pgsql/master/bin/pg_dump
--filter=/dev/stdin

It is working without any problem

Regards

Pavel

#52Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#48)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

čt 24. 9. 2020 v 19:47 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

rebase + minor change - using pg_get_line_buf instead pg_get_line_append

fresh rebase

Regards

Show quoted text

Pavel

Attachments:

pg_dump-filter-20201011.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20201011.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..068821583f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -751,6 +751,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign server),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_server
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c68db75b97..f1f9e8321e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -294,6 +296,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -367,6 +370,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -390,6 +394,7 @@ main(int argc, char **argv)
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -607,6 +612,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1035,6 +1044,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18632,3 +18643,160 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty rows */
+		if (*str == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
new file mode 100644
index 0000000000..957442c5e4
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -0,0 +1,129 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 22;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+t table_one\n";
+print $inputfile "+t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "-d table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-n public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+f doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+d sometable";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/include filter is not supported for this type of object/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-f someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not supported for this type of object/,
+	"broken format check");
#53Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#50)
Re: proposal: possibility to read dumped table's name from file

st 11. 11. 2020 v 6:32 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

út 10. 11. 2020 v 21:09 odesílatel Stephen Frost <sfrost@snowman.net>
napsal:

Greetings,

* Pavel Stehule (pavel.stehule@gmail.com) wrote:

rebase + minor change - using pg_get_line_buf instead pg_get_line_append

I started looking at this and went back through the thread and while I
tend to agree that JSON may not be a good choice for this, it's not the
only possible alternative. There is no doubt that pg_dump is already a
sophisticated data export tool, and likely to continue to gain new
features, such that having a configuration file for it would be very
handy, but this clearly isn't really going in a direction that would
allow for that.

Perhaps this feature could co-exist with a full blown configuration for
pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a newline
in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address these
issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters in
the future.

This is the correct argument - I will check a possibility to use strange
names, but there is the same possibility and functionality like we allow
from the command line. So you can use double quoted names. I'll check it.

Unlike for the pg_basebackup manifest, which we generate and read
entirely programatically, a config file for pg_dump would almost
certainly be updated manually (or, at least, parts of it would be and
perhaps other parts generated), which means it'd really be ideal to have
a proper way to support comments in it (something that the proposed
format also doesn't really get right- # must be the *first* character,
and you can only have whole-line comments..?), avoid extra unneeded
punctuation (or, at times, allow it- such as trailing commas in lists),
cleanly handle multi-line strings (consider the oft discussed idea
around having pg_dump support a WHERE clause for exporting data from
tables...), etc.

I think the proposed feature is very far to be the config file for pg_dump
(it implements a option "--filter"). This is not the target. It is not
designed for this. This is just an alternative for options like -t, -T, ...
and I am sure so nobody will generate this file manually. Main target of
this patch is eliminating problems with the max length of the command line.
So it is really not designed to be the config file for pg_dump.

Overall, -1 from me on this approach. Maybe it could be fixed up to
handle all the different names of objects that we support today
(something which, imv, is really a clear requirement for this feature to
be committed), but I suspect you'd end up half-way to yet another
configuration format when we could be working to support something like
TOML or maybe YAML... but if you want my 2c, TOML seems closer to what
we do for postgresql.conf and getting that over to something that's
standardized, while a crazy long shot, is a general nice idea, imv.

I have nothing against TOML, but I don't see a sense of usage in this
patch. This patch doesn't implement a config file for pg_dump, and I don't
see any sense or benefits of it. The TOML is designed for different
purposes. TOML is good for manual creating, but it is not this case.
Typical usage of this patch is some like, and TOML syntax (or JSON) is not
good for this.

psql -c "select '+t' || quote_ident(relname) from pg_class where relname
..." | pg_dump --filter=/dev/stdin

I can imagine some benefits of saved configure files for postgres
applications - but it should be designed generally and implemented
generally. Probably you would use one for pg_dump, psql, pg_restore, ....
But it is a different feature with different usage. This patch doesn't
implement option "--config", it implements option "--filter".

Some generic configuration for postgres binary applications is an
interesting idea. And TOML language can be well for this purpose. We can
parametrize applications by command line and by system variables. But
filtering objects is a really different case - although there is some small
intersection, and it will be used very differently, and I don't think so
one language can be practical for both cases. The object filtering is an
independent feature, and both features can coexist together.

Regards

Pavel

Show quoted text

Regards

Pavel

Thanks,

Stephen

#54Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Pavel Stehule (#50)
Re: proposal: possibility to read dumped table's name from file

On 2020-Nov-11, Pavel Stehule wrote:

I think the proposed feature is very far to be the config file for pg_dump
(it implements a option "--filter"). This is not the target. It is not
designed for this. This is just an alternative for options like -t, -T, ...
and I am sure so nobody will generate this file manually. Main target of
this patch is eliminating problems with the max length of the command line.
So it is really not designed to be the config file for pg_dump.

I agree that a replacement for existing command line arguments is a good
goal, but at the same time it's good to keep in mind the request that
more object types are supported as dumpable. While it's not necessary
that this infrastructure supports all object types in the first cut,
it'd be good to have it support that. I would propose that instead of a
single letter 't' etc we support keywords, maybe similar to those
returned by getObjectTypeDescription() (with additions -- for example
for "table data"). Then we can extend for other object types later
without struggling to find good letters for each.

Of course we could allow abbrevations for common cases, such that "t"
means "table".

For example: it'll be useful to support selective dumping of functions,
materialized views, foreign objects, etc.

#55Pavel Stehule
pavel.stehule@gmail.com
In reply to: Alvaro Herrera (#54)
Re: proposal: possibility to read dumped table's name from file

st 11. 11. 2020 v 16:17 odesílatel Alvaro Herrera <alvherre@alvh.no-ip.org>
napsal:

On 2020-Nov-11, Pavel Stehule wrote:

I think the proposed feature is very far to be the config file for

pg_dump

(it implements a option "--filter"). This is not the target. It is not
designed for this. This is just an alternative for options like -t, -T,

...

and I am sure so nobody will generate this file manually. Main target of
this patch is eliminating problems with the max length of the command

line.

So it is really not designed to be the config file for pg_dump.

I agree that a replacement for existing command line arguments is a good
goal, but at the same time it's good to keep in mind the request that
more object types are supported as dumpable. While it's not necessary
that this infrastructure supports all object types in the first cut,
it'd be good to have it support that. I would propose that instead of a
single letter 't' etc we support keywords, maybe similar to those
returned by getObjectTypeDescription() (with additions -- for example
for "table data"). Then we can extend for other object types later
without struggling to find good letters for each.

Of course we could allow abbrevations for common cases, such that "t"
means "table".

For example: it'll be useful to support selective dumping of functions,
materialized views, foreign objects, etc.

Implementation of this is trivial.

The hard work is mapping pg_dump options on database objects. t -> table is
simple, but n -> schema looks a little bit inconsistent - although it is
consistent with pg_dump. d or D - there is no system object like data. I am
afraid so there are two independent systems - pg_dump options, and database
objects, and it can be hard or not very practical to join these systems.
Unfortunately there is not good consistency in the short options of pg_dump
today. More - a lot of object names are multi words with inner space. This
is not too practical.

What about supporting two syntaxes?

1. first short current +-tndf filter - but the implementation should not be
limited to one char - there can be any string until space

2. long syntax - all these pg_dump options has long options, and then we
can extend this feature without any problem in future

table|exclude-table|exclude-table-data|schema|exclude-schema|include-foreign-data=PATTERN

so the content of filter file can looks like:

+t mytable
+t tabprefix*
-t bigtable

table=mytable2
exclude-table=mytable2

This format allows quick work for most common database objects, and it is
extensible and consistent with pg_dump's long options.

What do you think about it?

Personally, I am thinking that it is over-engineering a little bit, maybe
we can implement this feature just test first string after +- symbols
(instead first char like now) - and any enhanced syntax can be implemented
in future when there will be this requirement. Second syntax can be
implemented very simply, because it can be identified by first char
processing. We can implement second syntax only too. It will work too, but
I think so short syntax is more practical for daily work (for common
options). I expect so 99% percent of this objects will be "+t tablename".

Regards

Pavel

#56Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#51)
2 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On Wed, Nov 11, 2020 at 06:49:43AM +0100, Pavel Stehule wrote:

Perhaps this feature could co-exist with a full blown configuration for

pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a newline
in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address these
issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters in
the future.

I think it's a reasonable question - why would a new configuration file option
include support for only a handful of existing arguments but not the rest.

This is the correct argument - I will check a possibility to use strange
names, but there is the same possibility and functionality like we allow
from the command line. So you can use double quoted names. I'll check it.

I checked
echo "+t \"bad Name\"" | /usr/local/pgsql/master/bin/pg_dump --filter=/dev/stdin
It is working without any problem

I think it couldn't possibly work with newlines, since you call pg_get_line().
I realize that entering a newline into the shell would also be a PITA, but that
could be one *more* reason to support a config file - to allow terrible table
names to be in a file and avoid writing dash tee quote something enter else
quote in a pg_dump command, or shell script.

I fooled with argument parsing to handle reading from a file in the quickest
way. As written, this fails to handle multiple config files, and special table
names, which need to support arbitrary, logical lines, with quotes surrounding
newlines or other special chars. As written, the --config file is parsed
*after* all other arguments, so it could override previous args (like
--no-blobs --no-blogs, --file, --format, --compress, --lock-wait), which I
guess is bad, so the config file should be processed *during* argument parsing.
Unfortunately, I think that suggests duplicating parsing of all/most the
argument parsing for config file support - I'd be happy if someone suggested a
better way.

BTW, in your most recent patch:
s/empty rows/empty lines/
unbalanced parens: "invalid option type (use [+-]"

@cfbot: I renamed the patch so please ignore it.

--
Justin

Attachments:

0001-proposal-possibility-to-read-dumped-table-s-name-fro.patch.nottext/x-diff; charset=utf-8Download
From 8bff976d3d787898e71d3c8c893d5f9093268569 Mon Sep 17 00:00:00 2001
From: Pavel Stehule <pavel.stehule@gmail.com>
Date: Wed, 11 Nov 2020 06:54:22 +0100
Subject: [PATCH 1/2] proposal: possibility to read dumped table's name from
 file
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

--00000000000087718205b3ce6fcd
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi

=C4=8Dt 24. 9. 2020 v 19:47 odes=C3=ADlatel Pavel Stehule <pavel.stehule@gm=
ail.com>
napsal:

> Hi
>
> rebase + minor change - using pg_get_line_buf instead pg_get_line_append
>
>
fresh rebase

Regards
>

> Pavel
>

<div dir="ltr"><div>Hi<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">čt 24. 9. 2020 v 19:47 odesílatel Pavel Stehule &lt;<a href="mailto:pavel.stehule@gmail.com">pavel.stehule@gmail.com</a>&gt; napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi</div><div><br></div><div>rebase + minor change - using pg_get_line_buf instead pg_get_line_append</div><div><br></div></div></blockquote><div><br></div><div>fresh rebase</div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Regards <br></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Pavel<br></div></div>
</blockquote></div></div>
---
 doc/src/sgml/ref/pg_dump.sgml           |  50 +++++++
 src/bin/pg_dump/pg_dump.c               | 168 ++++++++++++++++++++++++
 src/bin/pg_dump/t/004_pg_dump_filter.pl | 129 ++++++++++++++++++
 3 files changed, 347 insertions(+)
 create mode 100644 src/bin/pg_dump/t/004_pg_dump_filter.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..068821583f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -751,6 +751,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign server),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_server
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c68db75b97..f1f9e8321e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -294,6 +296,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -367,6 +370,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -390,6 +394,7 @@ main(int argc, char **argv)
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -607,6 +612,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1035,6 +1044,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18632,3 +18643,160 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty rows */
+		if (*str == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-]",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
new file mode 100644
index 0000000000..957442c5e4
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -0,0 +1,129 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 22;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+t table_one\n";
+print $inputfile "+t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "-d table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-n public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+f doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+d sometable";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/include filter is not supported for this type of object/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-f someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not supported for this type of object/,
+	"broken format check");
-- 
2.17.0

0002-f-Read-additional-cmdline-args-from-config.patch.nottext/x-diff; charset=us-asciiDownload
From c91334a891490e0541522f0bf1872b4e7b45f802 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 16 Nov 2020 21:26:42 -0600
Subject: [PATCH 2/2] f!Read additional cmdline args from config

---
 src/bin/pg_dump/pg_dump.c               | 719 +++++++++++-------------
 src/bin/pg_dump/t/004_pg_dump_filter.pl |  50 +-
 2 files changed, 337 insertions(+), 432 deletions(-)

diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f1f9e8321e..00bbbbf5c6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -296,109 +296,38 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
-static void read_patterns_from_file(char *filename, DumpOptions *dopt);
-
+static void read_config_file(char *filename, DumpOptions *dopt);
+static void xgetopt(int argc, char **argv, DumpOptions *dopt);
+
+const char *filename = NULL;
+const char *format = "p";
+char	   *endptr;
+const char *dumpencoding = NULL;
+const char *dumpsnapshot = NULL;
+char	   *use_role = NULL;
+long		rowsPerInsert;
+int			numWorkers = 1;
+int			compressLevel = -1;
+int			plainText = 0;
+bool		g_verbose = false;
+char		*configfile = NULL;
+
+static DumpOptions dopt;
 
 int
 main(int argc, char **argv)
 {
-	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
 	int			numObjs;
 	DumpableObject *boundaryObjs;
 	int			i;
-	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
-	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
 
-	static DumpOptions dopt;
-
-	static struct option long_options[] = {
-		{"data-only", no_argument, NULL, 'a'},
-		{"blobs", no_argument, NULL, 'b'},
-		{"no-blobs", no_argument, NULL, 'B'},
-		{"clean", no_argument, NULL, 'c'},
-		{"create", no_argument, NULL, 'C'},
-		{"dbname", required_argument, NULL, 'd'},
-		{"file", required_argument, NULL, 'f'},
-		{"format", required_argument, NULL, 'F'},
-		{"host", required_argument, NULL, 'h'},
-		{"jobs", 1, NULL, 'j'},
-		{"no-reconnect", no_argument, NULL, 'R'},
-		{"no-owner", no_argument, NULL, 'O'},
-		{"port", required_argument, NULL, 'p'},
-		{"schema", required_argument, NULL, 'n'},
-		{"exclude-schema", required_argument, NULL, 'N'},
-		{"schema-only", no_argument, NULL, 's'},
-		{"superuser", required_argument, NULL, 'S'},
-		{"table", required_argument, NULL, 't'},
-		{"exclude-table", required_argument, NULL, 'T'},
-		{"no-password", no_argument, NULL, 'w'},
-		{"password", no_argument, NULL, 'W'},
-		{"username", required_argument, NULL, 'U'},
-		{"verbose", no_argument, NULL, 'v'},
-		{"no-privileges", no_argument, NULL, 'x'},
-		{"no-acl", no_argument, NULL, 'x'},
-		{"compress", required_argument, NULL, 'Z'},
-		{"encoding", required_argument, NULL, 'E'},
-		{"help", no_argument, NULL, '?'},
-		{"version", no_argument, NULL, 'V'},
-
-		/*
-		 * the following options don't have an equivalent short option letter
-		 */
-		{"attribute-inserts", no_argument, &dopt.column_inserts, 1},
-		{"binary-upgrade", no_argument, &dopt.binary_upgrade, 1},
-		{"column-inserts", no_argument, &dopt.column_inserts, 1},
-		{"disable-dollar-quoting", no_argument, &dopt.disable_dollar_quoting, 1},
-		{"disable-triggers", no_argument, &dopt.disable_triggers, 1},
-		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
-		{"exclude-table-data", required_argument, NULL, 4},
-		{"extra-float-digits", required_argument, NULL, 8},
-		{"filter", required_argument, NULL, 12},
-		{"if-exists", no_argument, &dopt.if_exists, 1},
-		{"inserts", no_argument, NULL, 9},
-		{"lock-wait-timeout", required_argument, NULL, 2},
-		{"no-tablespaces", no_argument, &dopt.outputNoTablespaces, 1},
-		{"quote-all-identifiers", no_argument, &quote_all_identifiers, 1},
-		{"load-via-partition-root", no_argument, &dopt.load_via_partition_root, 1},
-		{"role", required_argument, NULL, 3},
-		{"section", required_argument, NULL, 5},
-		{"serializable-deferrable", no_argument, &dopt.serializable_deferrable, 1},
-		{"snapshot", required_argument, NULL, 6},
-		{"strict-names", no_argument, &strict_names, 1},
-		{"use-set-session-authorization", no_argument, &dopt.use_setsessauth, 1},
-		{"no-comments", no_argument, &dopt.no_comments, 1},
-		{"no-publications", no_argument, &dopt.no_publications, 1},
-		{"no-security-labels", no_argument, &dopt.no_security_labels, 1},
-		{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
-		{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
-		{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
-		{"no-sync", no_argument, NULL, 7},
-		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
-		{"rows-per-insert", required_argument, NULL, 10},
-		{"include-foreign-data", required_argument, NULL, 11},
-		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
-		{"include-foreign-data-file", required_argument, NULL, 17},
-
-		{NULL, 0, NULL, 0}
-	};
-
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -427,200 +356,7 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
-							long_options, &optindex)) != -1)
-	{
-		switch (c)
-		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			case 12:			/* filter implementation */
-				read_patterns_from_file(optarg, &dopt);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
-				exit_nicely(1);
-		}
-	}
+	xgetopt(argc, argv, &dopt);
 
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
@@ -639,6 +375,9 @@ main(int argc, char **argv)
 		exit_nicely(1);
 	}
 
+	if (configfile)
+		read_config_file(configfile, &dopt);
+
 	/* --column-inserts implies --inserts */
 	if (dopt.column_inserts && dopt.dump_inserts == 0)
 		dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
@@ -1000,6 +739,280 @@ main(int argc, char **argv)
 	exit_nicely(0);
 }
 
+static struct option long_options[] = {
+	{"data-only", no_argument, NULL, 'a'},
+	{"blobs", no_argument, NULL, 'b'},
+	{"no-blobs", no_argument, NULL, 'B'},
+	{"clean", no_argument, NULL, 'c'},
+	{"create", no_argument, NULL, 'C'},
+	{"dbname", required_argument, NULL, 'd'},
+	{"file", required_argument, NULL, 'f'},
+	{"format", required_argument, NULL, 'F'},
+	{"host", required_argument, NULL, 'h'},
+	{"jobs", 1, NULL, 'j'},
+	{"no-reconnect", no_argument, NULL, 'R'},
+	{"no-owner", no_argument, NULL, 'O'},
+	{"port", required_argument, NULL, 'p'},
+	{"schema", required_argument, NULL, 'n'},
+	{"exclude-schema", required_argument, NULL, 'N'},
+	{"schema-only", no_argument, NULL, 's'},
+	{"superuser", required_argument, NULL, 'S'},
+	{"table", required_argument, NULL, 't'},
+	{"exclude-table", required_argument, NULL, 'T'},
+	{"no-password", no_argument, NULL, 'w'},
+	{"password", no_argument, NULL, 'W'},
+	{"username", required_argument, NULL, 'U'},
+	{"verbose", no_argument, NULL, 'v'},
+	{"no-privileges", no_argument, NULL, 'x'},
+	{"no-acl", no_argument, NULL, 'x'},
+	{"compress", required_argument, NULL, 'Z'},
+	{"encoding", required_argument, NULL, 'E'},
+	{"help", no_argument, NULL, '?'},
+	{"version", no_argument, NULL, 'V'},
+
+	/*
+	 * the following options don't have an equivalent short option letter
+	 */
+	{"attribute-inserts", no_argument, &dopt.column_inserts, 1},
+	{"binary-upgrade", no_argument, &dopt.binary_upgrade, 1},
+	{"column-inserts", no_argument, &dopt.column_inserts, 1},
+	{"disable-dollar-quoting", no_argument, &dopt.disable_dollar_quoting, 1},
+	{"disable-triggers", no_argument, &dopt.disable_triggers, 1},
+	{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
+	{"exclude-table-data", required_argument, NULL, 4},
+	{"extra-float-digits", required_argument, NULL, 8},
+	// {"config", required_argument, NULL, 12},
+	{"config", required_argument, NULL, 12},
+	{"if-exists", no_argument, &dopt.if_exists, 1},
+	{"inserts", no_argument, NULL, 9},
+	{"lock-wait-timeout", required_argument, NULL, 2},
+	{"no-tablespaces", no_argument, &dopt.outputNoTablespaces, 1},
+	{"quote-all-identifiers", no_argument, &quote_all_identifiers, 1},
+	{"load-via-partition-root", no_argument, &dopt.load_via_partition_root, 1},
+	{"role", required_argument, NULL, 3},
+	{"section", required_argument, NULL, 5},
+	{"serializable-deferrable", no_argument, &dopt.serializable_deferrable, 1},
+	{"snapshot", required_argument, NULL, 6},
+	{"strict-names", no_argument, &strict_names, 1},
+	{"use-set-session-authorization", no_argument, &dopt.use_setsessauth, 1},
+	{"no-comments", no_argument, &dopt.no_comments, 1},
+	{"no-publications", no_argument, &dopt.no_publications, 1},
+	{"no-security-labels", no_argument, &dopt.no_security_labels, 1},
+	{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
+	{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
+	{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+	{"no-sync", no_argument, NULL, 7},
+	{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+	{"rows-per-insert", required_argument, NULL, 10},
+	{"include-foreign-data", required_argument, NULL, 11},
+	{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+	{"include-foreign-data-file", required_argument, NULL, 17},
+
+	{NULL, 0, NULL, 0}
+};
+
+static void
+xgetopt(int argc, char **argv, DumpOptions *dopt)
+{
+	int			c;
+	int			optindex;
+
+	while ((c = getopt_long(argc, argv, "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+							long_options, &optindex)) != -1)
+	{
+		switch (c)
+		{
+			case 'a':			/* Dump data only */
+				dopt->dataOnly = true;
+				break;
+
+			case 'b':			/* Dump blobs */
+				dopt->outputBlobs = true;
+				break;
+
+			case 'B':			/* Don't dump blobs */
+				dopt->dontOutputBlobs = true;
+				break;
+
+			case 'c':			/* clean (i.e., drop) schema prior to create */
+				dopt->outputClean = 1;
+				break;
+
+			case 'C':			/* Create DB */
+				dopt->outputCreateDB = 1;
+				break;
+
+			case 'd':			/* database name */
+				dopt->cparams.dbname = pg_strdup(optarg);
+				break;
+
+			case 'E':			/* Dump encoding */
+				dumpencoding = pg_strdup(optarg);
+				break;
+
+			case 'f':
+				filename = pg_strdup(optarg);
+				break;
+
+			case 'F':
+				format = pg_strdup(optarg);
+				break;
+
+			case 'h':			/* server host */
+				dopt->cparams.pghost = pg_strdup(optarg);
+				break;
+
+			case 'j':			/* number of dump jobs */
+				numWorkers = atoi(optarg);
+				break;
+
+			case 'n':			/* include schema(s) */
+				simple_string_list_append(&schema_include_patterns, optarg);
+				dopt->include_everything = false;
+				break;
+
+			case 'N':			/* exclude schema(s) */
+				simple_string_list_append(&schema_exclude_patterns, optarg);
+				break;
+
+			case 'O':			/* Don't reconnect to match owner */
+				dopt->outputNoOwner = 1;
+				break;
+
+			case 'p':			/* server port */
+				dopt->cparams.pgport = pg_strdup(optarg);
+				break;
+
+			case 'R':
+				/* no-op, still accepted for backwards compatibility */
+				break;
+
+			case 's':			/* dump schema only */
+				dopt->schemaOnly = true;
+				break;
+
+			case 'S':			/* Username for superuser in plain text output */
+				dopt->outputSuperuser = pg_strdup(optarg);
+				break;
+
+			case 't':			/* include table(s) */
+				simple_string_list_append(&table_include_patterns, optarg);
+				dopt->include_everything = false;
+				break;
+
+			case 'T':			/* exclude table(s) */
+				simple_string_list_append(&table_exclude_patterns, optarg);
+				break;
+
+			case 'U':
+				dopt->cparams.username = pg_strdup(optarg);
+				break;
+
+			case 'v':			/* verbose */
+				g_verbose = true;
+				pg_logging_increase_verbosity();
+				break;
+
+			case 'w':
+				dopt->cparams.promptPassword = TRI_NO;
+				break;
+
+			case 'W':
+				dopt->cparams.promptPassword = TRI_YES;
+				break;
+
+			case 'x':			/* skip ACL dump */
+				dopt->aclsSkip = true;
+				break;
+
+			case 'Z':			/* Compression Level */
+				compressLevel = atoi(optarg);
+				if (compressLevel < 0 || compressLevel > 9)
+				{
+					pg_log_error("compression level must be in range 0..9");
+					exit_nicely(1);
+				}
+				break;
+
+			case 0:
+				/* This covers the long options. */
+				break;
+
+			case 2:				/* lock-wait-timeout */
+				dopt->lockWaitTimeout = pg_strdup(optarg);
+				break;
+
+			case 3:				/* SET ROLE */
+				use_role = pg_strdup(optarg);
+				break;
+
+			case 4:				/* exclude table(s) data */
+				simple_string_list_append(&tabledata_exclude_patterns, optarg);
+				break;
+
+			case 5:				/* section */
+				set_dump_section(optarg, &dopt->dumpSections);
+				break;
+
+			case 6:				/* snapshot */
+				dumpsnapshot = pg_strdup(optarg);
+				break;
+
+			case 7:				/* no-sync */
+				dosync = false;
+				break;
+
+			case 8:
+				have_extra_float_digits = true;
+				extra_float_digits = atoi(optarg);
+				if (extra_float_digits < -15 || extra_float_digits > 3)
+				{
+					pg_log_error("extra_float_digits must be in range -15..3");
+					exit_nicely(1);
+				}
+				break;
+
+			case 9:				/* inserts */
+
+				/*
+				 * dump_inserts also stores --rows-per-insert, careful not to
+				 * overwrite that.
+				 */
+				if (dopt->dump_inserts == 0)
+					dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+				break;
+
+			case 10:			/* rows per insert */
+				errno = 0;
+				rowsPerInsert = strtol(optarg, &endptr, 10);
+
+				if (endptr == optarg || *endptr != '\0' ||
+					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+					errno == ERANGE)
+				{
+					pg_log_error("rows-per-insert must be in range %d..%d",
+								 1, INT_MAX);
+					exit_nicely(1);
+				}
+				dopt->dump_inserts = (int) rowsPerInsert;
+				break;
+
+			case 11:			/* include foreign data */
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  optarg);
+				break;
+
+			case 12:			/* config implementation */
+				// read_config_file(optarg, dopt);
+				configfile = optarg;
+				break;
+
+			default:
+				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+				exit_nicely(1);
+		}
+	}
+}
 
 static void
 help(const char *progname)
@@ -1044,8 +1057,7 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
-	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
-			 "                               from the filter file\n"));
+	printf(_("  --config=FILENAME            read additional command line argument config from file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18644,32 +18656,10 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 		pg_log_warning("could not parse reloptions array");
 }
 
-/*
- * Print error message and exit.
- */
 static void
-exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
-{
-	pg_log_error("invalid format of filter file \"%s\": %s",
-				 filename,
-				 message);
-
-	fprintf(stderr, "%d: %s\n", lineno, line);
-
-	if (fp != stdin)
-		fclose(fp);
-
-	exit_nicely(-1);
-}
-
-/*
- * Read dumped object specification from file
- */
-static void
-read_patterns_from_file(char *filename, DumpOptions *dopt)
+read_config_file(char *filename, DumpOptions *dopt)
 {
 	FILE	   *fp;
-	int			lineno = 0;
 	StringInfoData line;
 
 	/* use "-" as symbol for stdin */
@@ -18685,111 +18675,44 @@ read_patterns_from_file(char *filename, DumpOptions *dopt)
 
 	initStringInfo(&line);
 
-	while (pg_get_line_buf(fp, &line))
+	while (pg_get_line_buf(fp, &line)) // XXX: need to read a "logical" line
 	{
-		bool		is_include;
-		char		objecttype;
-		char	   *objectname;
-		char	   *str = line.data;
-
-		lineno += 1;
+		char	*str = line.data;
+		int		argc;
+		char	*argv[] = {"pg_dump", str, NULL, NULL};
+		char	*argv2;
 
 		(void) pg_strip_crlf(str);
 
-		/* ignore empty rows */
+		/* ignore empty lines */
 		if (*str == '\0')
 			continue;
 
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
 		/* when first char is hash, ignore whole line */
 		if (*str == '#')
 			continue;
 
-		if (str[1] == '\0')
-			exit_invalid_filter_format(fp,
-									   filename,
-									   "line too short",
-									   str,
-									   lineno);
-
-		if (str[0] == '+')
-			is_include = true;
-		else if (str[0] == '-')
-			is_include = false;
-		else
-			exit_invalid_filter_format(fp,
-									   filename,
-									   "invalid option type (use [+-]",
-									   str,
-									   lineno);
-
-		objecttype = str[1];
-		objectname = &str[2];
-
-		/* skip initial spaces */
-		while (isspace(*objectname))
-			objectname++;
+		/* Not allowed to set dbname */
+		if (*str != '-')
+			fatal("invalid line in file \"%s\": %s", filename, str);
 
-		if (*objectname == '\0')
-			exit_invalid_filter_format(fp,
-									   filename,
-									   "missing object name",
-									   str,
-									   lineno);
-
-		if (objecttype == 't')
-		{
-			if (is_include)
-			{
-				simple_string_list_append(&table_include_patterns,
-										  objectname);
-				dopt->include_everything = false;
-			}
-			else
-				simple_string_list_append(&table_exclude_patterns,
-										  objectname);
-		}
-		else if (objecttype == 'n')
-		{
-			if (is_include)
-			{
-				simple_string_list_append(&schema_include_patterns,
-										  objectname);
-				dopt->include_everything = false;
-			}
-			else
-				simple_string_list_append(&schema_exclude_patterns,
-										  objectname);
-		}
-		else if (objecttype == 'd')
-		{
-			if (is_include)
-				exit_invalid_filter_format(fp,
-										   filename,
-										   "include filter is not supported for this type of object",
-										   str,
-										   lineno);
-			else
-				simple_string_list_append(&tabledata_exclude_patterns,
-										  objectname);
-		}
-		else if (objecttype == 'f')
+		argv2 = strchr(str, ' ');
+		if (argv2 == NULL)
+			argc = 2;
+		else
 		{
-			if (is_include)
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  objectname);
-			else
-				exit_invalid_filter_format(fp,
-										   filename,
-										   "exclude filter is not supported for this type of object",
-										   str,
-										   lineno);
+			argc = 3;
+			*argv2 = '\0';
+			argv[2] = argv2 + 1;
+			// fprintf(stderr, "passing argv %s..%s\n", argv[1], argv[2]);
 		}
-		else
-			exit_invalid_filter_format(fp,
-									   filename,
-									   "invalid object type (use [tndf])",
-									   str,
-									   lineno);
+
+		optind = 0; /* Reset it */
+		xgetopt(argc, argv, dopt);
 	}
 
 	pfree(line.data);
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
index 957442c5e4..77632a7b99 100644
--- a/src/bin/pg_dump/t/004_pg_dump_filter.pl
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -4,7 +4,7 @@ use warnings;
 use Config;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 22;
+use Test::More tests => 18;
 
 my $tempdir       = TestLib::tempdir;
 my $inputfile;
@@ -27,16 +27,16 @@ $node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE **
 
 open $inputfile, '>', "$tempdir/inputfile.txt";
 
-print $inputfile "+t table_one\n";
-print $inputfile "+t table_two\n";
+print $inputfile "-t table_one\n";
+print $inputfile "-t table_two\n";
 print $inputfile "# skip this line\n";
 print $inputfile "\n";
-print $inputfile "-d table_one\n";
+print $inputfile "--exclude-table-data table_one\n";
 close $inputfile;
 
 my ($cmd, $stdout, $stderr, $result);
 command_ok(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--config=$tempdir/inputfile.txt", 'postgres' ],
 	"dump tables with filter");
 
 my $dump = slurp_file($plainfile);
@@ -49,11 +49,11 @@ ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
 
 open $inputfile, '>', "$tempdir/inputfile.txt";
 
-print $inputfile "-t table_one\n";
+print $inputfile "-T table_one\n";
 close $inputfile;
 
 command_ok(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--config=$tempdir/inputfile.txt", 'postgres' ],
 	"dump tables with filter");
 
 $dump = slurp_file($plainfile);
@@ -64,11 +64,11 @@ ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
 
 open $inputfile, '>', "$tempdir/inputfile.txt";
 
-print $inputfile "-n public\n";
+print $inputfile "-N public\n";
 close $inputfile;
 
 command_ok(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--config=$tempdir/inputfile.txt", 'postgres' ],
 	"dump tables with filter");
 
 $dump = slurp_file($plainfile);
@@ -81,11 +81,11 @@ ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
 
 open $inputfile, '>', "$tempdir/inputfile.txt";
 
-print $inputfile "+f doesnt_exists\n";
+print $inputfile "--include-foreign-data=doesnt_exists\n";
 close $inputfile;
 
 command_fails_like(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--config=$tempdir/inputfile.txt", 'postgres' ],
 	qr/pg_dump: error: no matching foreign servers were found for pattern/,
 	"dump foreign server");
 
@@ -97,33 +97,15 @@ print $inputfile "k";
 close $inputfile;
 
 command_fails_like(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
-	qr/pg_dump: error: invalid format of filter file/,
-	"broken format check");
-
-open $inputfile, '>', "$tempdir/inputfile.txt";
-print $inputfile "+";
-close $inputfile;
-
-command_fails_like(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
-	qr/pg_dump: error: invalid format of filter file/,
-	"broken format check");
-
-open $inputfile, '>', "$tempdir/inputfile.txt";
-print $inputfile "+d sometable";
-close $inputfile;
-
-command_fails_like(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
-	qr/include filter is not supported for this type of object/,
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--config=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid line in file "[^"]*": k/,
 	"broken format check");
 
 open $inputfile, '>', "$tempdir/inputfile.txt";
-print $inputfile "-f someforeignserver";
+print $inputfile "-k";
 close $inputfile;
 
 command_fails_like(
-	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
-	qr/exclude filter is not supported for this type of object/,
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--config=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: invalid option -- 'k'/,
 	"broken format check");
-- 
2.17.0

#57Stephen Frost
sfrost@snowman.net
In reply to: Justin Pryzby (#56)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Justin Pryzby (pryzby@telsasoft.com) wrote:

On Wed, Nov 11, 2020 at 06:49:43AM +0100, Pavel Stehule wrote:

Perhaps this feature could co-exist with a full blown configuration for
pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a newline
in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address these
issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters in
the future.

I think it's a reasonable question - why would a new configuration file option
include support for only a handful of existing arguments but not the rest.

Even if the first version of having a config file for pg_dump only
supported some options, that would be reasonable imv, but I dislike the
idea of building it in such a way that it'll be awkward to add more
options to it in the future, something that I definitely think people
would like to see (I know I would...).

This is the correct argument - I will check a possibility to use strange
names, but there is the same possibility and functionality like we allow
from the command line. So you can use double quoted names. I'll check it.

I checked
echo "+t \"bad Name\"" | /usr/local/pgsql/master/bin/pg_dump --filter=/dev/stdin
It is working without any problem

I think it couldn't possibly work with newlines, since you call pg_get_line().

Yeah, I didn't really believe that it actually worked but hadn't had a
chance to demonstrate that it didn't yet.

I realize that entering a newline into the shell would also be a PITA, but that
could be one *more* reason to support a config file - to allow terrible table
names to be in a file and avoid writing dash tee quote something enter else
quote in a pg_dump command, or shell script.

Agreed.

I fooled with argument parsing to handle reading from a file in the quickest
way. As written, this fails to handle multiple config files, and special table
names, which need to support arbitrary, logical lines, with quotes surrounding
newlines or other special chars. As written, the --config file is parsed
*after* all other arguments, so it could override previous args (like
--no-blobs --no-blogs, --file, --format, --compress, --lock-wait), which I
guess is bad, so the config file should be processed *during* argument parsing.
Unfortunately, I think that suggests duplicating parsing of all/most the
argument parsing for config file support - I'd be happy if someone suggested a
better way.

This still feels like we're trying to quickly hack-and-slash at adding a
config file option rather than thinking through what a sensible design
for a pg_dump config file would look like. Having a way to avoid having
multiple places in the code that has to handle all the possible options
is a nice idea but, as I tried to allude to up-thread, I fully expect
that once we've got this config file capability that we're going to want
to add things to it that would be difficult to utilize through the
command-line and so I expect these code paths to diverge anyway.

I would imagine something like:

[connection]
db-host=whatever
db-port=5433
...

[output]
owners = true
privileges = false
format = "custom"
file = "myoutput.dump"

# This is a comment
[include]
tables = [ "sometable", "table with spaces",
"table with quoted \"",
"""this is my table
with a carriage return""", "anothertable" ]
table-patterns = [ "table*" ]
schemas = [ "myschema" ]

[exclude]
tables = [ "similar to include" ]
functions = [ "somefunction(int)" ]

etc, etc ...

Thanks,

Stephen

#58Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#56)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

út 17. 11. 2020 v 22:53 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Wed, Nov 11, 2020 at 06:49:43AM +0100, Pavel Stehule wrote:

Perhaps this feature could co-exist with a full blown configuration for

pg_dump, but even then there's certainly issues with what's proposed-
how would you handle explicitly asking for a table which is named
" mytable" to be included or excluded? Or a table which has a

newline

in it? Using a standardized format which supports the full range of
what we do in a table name, explicitly and clearly, would address

these

issues and also give us the flexibility to extend the options which
could be used through the configuration file beyond just the filters

in

the future.

I think it's a reasonable question - why would a new configuration file
option
include support for only a handful of existing arguments but not the rest.

I don't see a strong technical problem - enhancing parsing is not hard
work, but I miss a use case for this. The option "--filter" tries to solve
a problem with limited command line size. This is a clean use case and
there and supported options are options that can be used repeatedly on the
command line. Nothing less, nothing more. The format that is used is
designed just for this purpose.

When we would implement an alternative configuration to command line and
system environments, then the use case should be defined first. When the
use case is defined, we can talk about implementation and about good
format. There are a lot of interesting formats, but I miss a reason why the
usage of this alternative configuration can be helpful for pg_dump. Using
external libraries for richer formats means a new dependency, necessity to
solve portability issues, and maybe other issues, and for this there should
be a good use case. Passing a list of tables for dumping doesn't need a
rich format.

I cannot imagine using a config file with generated object names and some
other options together. Maybe if these configurations will not be too long
(then handy written) configuration can be usable. But when I think about
using pg_dump from some bash scripts, then much more practical is using
usual command line options and passing a list of objects by pipe. I really
miss the use case for special pg_dump's config file, and if there is, then
it is very different from a use case for "--filter" option.

This is the correct argument - I will check a possibility to use

strange

names, but there is the same possibility and functionality like we

allow

from the command line. So you can use double quoted names. I'll check

it.

I checked
echo "+t \"bad Name\"" | /usr/local/pgsql/master/bin/pg_dump

--filter=/dev/stdin

It is working without any problem

I think it couldn't possibly work with newlines, since you call
pg_get_line().
I realize that entering a newline into the shell would also be a PITA, but
that
could be one *more* reason to support a config file - to allow terrible
table
names to be in a file and avoid writing dash tee quote something enter else
quote in a pg_dump command, or shell script.

New patch is working with names that contains multilines

[pavel@localhost postgresql.master]$ psql -At -X -c "select '+t ' ||
quote_ident(table_name) from information_schema.tables where table_name
like 'foo%'"| /usr/local/pgsql/master/bin/pg_dump --filter=/dev/stdin
--
-- PostgreSQL database dump
--

-- Dumped from database version 14devel
-- Dumped by pg_dump version 14devel

-
-- Name: foo boo; Type: TABLE; Schema: public; Owner: pavel
--

CREATE TABLE public."foo
boo" (
a integer
);

ALTER TABLE public."foo
boo" OWNER TO pavel;

--
-- Data for Name: foo boo; Type: TABLE DATA; Schema: public; Owner: pavel
--

COPY public."foo
boo" (a) FROM stdin;
\.

--
-- PostgreSQL database dump complete
--

I fooled with argument parsing to handle reading from a file in the
quickest
way. As written, this fails to handle multiple config files, and special
table
names, which need to support arbitrary, logical lines, with quotes
surrounding
newlines or other special chars. As written, the --config file is parsed
*after* all other arguments, so it could override previous args (like
--no-blobs --no-blogs, --file, --format, --compress, --lock-wait), which I
guess is bad, so the config file should be processed *during* argument
parsing.
Unfortunately, I think that suggests duplicating parsing of all/most the
argument parsing for config file support - I'd be happy if someone
suggested a
better way.

BTW, in your most recent patch:
s/empty rows/empty lines/
unbalanced parens: "invalid option type (use [+-]"

should be fixed now, thank you for check

Regards

Pavel

Show quoted text

@cfbot: I renamed the patch so please ignore it.

--
Justin

Attachments:

pg_dump-filter-option-20201119.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-option-20201119.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..068821583f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -751,6 +751,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign server),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_server
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dc1d41dd8d..074d57ad3c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -294,6 +296,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -367,6 +370,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -390,6 +394,7 @@ main(int argc, char **argv)
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -607,6 +612,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1035,6 +1044,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18635,3 +18646,219 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	PQExpBuffer quoted_name = NULL;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty lines */
+		if (*str == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-])",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (*objectname == '"')
+		{
+			PQExpBuffer		quoted_name;
+			char	   *ptr = objectname + 1;
+
+			quoted_name = createPQExpBuffer();
+
+			appendPQExpBufferChar(quoted_name, '"');
+
+			while (1)
+			{
+				if (*ptr == '\0')
+				{
+					if (!pg_get_line_buf(fp, &line))
+						exit_invalid_filter_format(fp,
+												   filename,
+												   "unexpected end of file",
+												   "",
+												   lineno);
+
+					if (ferror(fp))
+						fatal("could not read from file \"%s\": %m", filename);
+
+					appendPQExpBufferChar(quoted_name, '\n');
+					ptr = line.data;
+					lineno += 1;
+				}
+
+				appendPQExpBufferChar(quoted_name, *ptr);
+				if (*ptr++ == '"')
+				{
+					if (*ptr == '"')
+						ptr++;
+					else
+						break;
+				}
+			}
+
+			/* check garbage after identifier */
+			while (*ptr != '\0')
+			{
+				if (!isspace(*ptr))
+					exit_invalid_filter_format(fp,
+											   filename,
+											   "unexpected chars after object name",
+											   ptr,
+											   lineno);
+				ptr++;
+			}
+
+			objectname = quoted_name->data;
+		}
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+		if (quoted_name)
+		{
+			destroyPQExpBuffer(quoted_name);
+			quoted_name = NULL;
+		}
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
new file mode 100644
index 0000000000..957442c5e4
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -0,0 +1,129 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 22;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+t table_one\n";
+print $inputfile "+t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "-d table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-n public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+f doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+d sometable";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/include filter is not supported for this type of object/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-f someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not supported for this type of object/,
+	"broken format check");
#59Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#58)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

BTW, in your most recent patch:
s/empty rows/empty lines/
unbalanced parens: "invalid option type (use [+-]"

should be fixed now, thank you for check

minor update - fixed handling of processing names with double quotes inside

Show quoted text

Regards

Pavel

@cfbot: I renamed the patch so please ignore it.

--
Justin

Attachments:

pg_dump-filter-option-20201119-2.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-option-20201119-2.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..068821583f 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -751,6 +751,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(+|-)[tnfd] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first character specifies whether the object is to be included
+        (<literal>+</literal>) or excluded (<literal>-</literal>), and the
+        second character specifies the type of object to be filtered:
+        <literal>t</literal> (table),
+        <literal>n</literal> (schema),
+        <literal>f</literal> (foreign server),
+        <literal>d</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
++t mytable1
++f some_foreign_server
+-d mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dc1d41dd8d..e2567ca606 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -294,6 +296,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -367,6 +370,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -390,6 +394,7 @@ main(int argc, char **argv)
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -607,6 +612,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1035,6 +1044,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18635,3 +18646,219 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	PQExpBuffer quoted_name = NULL;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore empty lines */
+		if (*str == '\0')
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		if (str[1] == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "line too short",
+									   str,
+									   lineno);
+
+		if (str[0] == '+')
+			is_include = true;
+		else if (str[0] == '-')
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid option type (use [+-])",
+									   str,
+									   lineno);
+
+		objecttype = str[1];
+		objectname = &str[2];
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (*objectname == '"')
+		{
+			PQExpBuffer		quoted_name;
+			char	   *ptr = objectname + 1;
+
+			quoted_name = createPQExpBuffer();
+
+			appendPQExpBufferChar(quoted_name, '"');
+
+			while (1)
+			{
+				if (*ptr == '\0')
+				{
+					if (!pg_get_line_buf(fp, &line))
+						exit_invalid_filter_format(fp,
+												   filename,
+												   "unexpected end of file",
+												   "",
+												   lineno);
+
+					if (ferror(fp))
+						fatal("could not read from file \"%s\": %m", filename);
+
+					appendPQExpBufferChar(quoted_name, '\n');
+					ptr = line.data;
+					lineno += 1;
+				}
+
+				appendPQExpBufferChar(quoted_name, *ptr);
+				if (*ptr++ == '"')
+				{
+					if (*ptr == '"')
+						appendPQExpBufferChar(quoted_name, *ptr++);
+					else
+						break;
+				}
+			}
+
+			/* check garbage after identifier */
+			while (*ptr != '\0')
+			{
+				if (!isspace(*ptr))
+					exit_invalid_filter_format(fp,
+											   filename,
+											   "unexpected chars after object name",
+											   ptr,
+											   lineno);
+				ptr++;
+			}
+
+			objectname = quoted_name->data;
+		}
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'n')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "invalid object type (use [tndf])",
+									   str,
+									   lineno);
+		if (quoted_name)
+		{
+			destroyPQExpBuffer(quoted_name);
+			quoted_name = NULL;
+		}
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filter.pl b/src/bin/pg_dump/t/004_pg_dump_filter.pl
new file mode 100644
index 0000000000..957442c5e4
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filter.pl
@@ -0,0 +1,129 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 22;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+t table_one\n";
+print $inputfile "+t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "-d table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-n public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "+f doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid format of filter file/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "+d sometable";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/include filter is not supported for this type of object/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-f someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not supported for this type of object/,
+	"broken format check");
#60Dean Rasheed
dean.a.rasheed@gmail.com
In reply to: Pavel Stehule (#59)
Re: proposal: possibility to read dumped table's name from file

On Thu, 19 Nov 2020 at 19:57, Pavel Stehule <pavel.stehule@gmail.com> wrote:

minor update - fixed handling of processing names with double quotes inside

I see this is marked RFC, but reading the thread it doesn't feel like
we have reached consensus on the design for this feature.

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

IMO, a pg_dump config file should be able to specify all options
currently supported through the command line, and vice versa (subject
to command line length limits), with a single common code path for
handling options. That way, any new options we add will work on the
command line and in config files. Likewise, the user should only need
to learn one set of options, and have the choice of specifying them on
the command line or in a config file (or a mix of both).

I can imagine eventually supporting multiple different file formats,
each just being a different representation of the same data, so
perhaps this could work with 2 new options:

--option-file-format=plain|yaml|json|...
--option-file=filename

with "plain" being the default initial implementation, which might be
something like our current postgresql.conf file format.

Also, I think we should allow multiple "--option-file" arguments
(e.g., to list different object types in different files), and for a
config file to contain its own "--option-file" arguments, to allow
config files to include other config files.

The current design feels far too limited to me, and requires new code
and new syntax to be added each time we extend it, so I'm -1 on this
patch as it stands.

Regards,
Dean

#61Pavel Stehule
pavel.stehule@gmail.com
In reply to: Dean Rasheed (#60)
Re: proposal: possibility to read dumped table's name from file

Hi

st 25. 11. 2020 v 19:25 odesílatel Dean Rasheed <dean.a.rasheed@gmail.com>
napsal:

On Thu, 19 Nov 2020 at 19:57, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

minor update - fixed handling of processing names with double quotes

inside

I see this is marked RFC, but reading the thread it doesn't feel like
we have reached consensus on the design for this feature.

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

Nobody sent a real use case for introducing the config file. There was a
discussion about formats, and you introduce other dimensions and
variability.

But I don't understand why? What is a use case? What is a benefit against
command line, or libpq variables? And why should config files be better as
a solution for limited length of command line, when I need to dump
thousands of tables exactly specified?

Regards

Pavel

IMO, a pg_dump config file should be able to specify all options
currently supported through the command line, and vice versa (subject
to command line length limits), with a single common code path for
handling options. That way, any new options we add will work on the
command line and in config files. Likewise, the user should only need
to learn one set of options, and have the choice of specifying them on
the command line or in a config file (or a mix of both).

I can imagine eventually supporting multiple different file formats,
each just being a different representation of the same data, so
perhaps this could work with 2 new options:

--option-file-format=plain|yaml|json|...
--option-file=filename

with "plain" being the default initial implementation, which might be
something like our current postgresql.conf file format.

Also, I think we should allow multiple "--option-file" arguments
(e.g., to list different object types in different files), and for a
config file to contain its own "--option-file" arguments, to allow
config files to include other config files.

The current design feels far too limited to me, and requires new code
and new syntax to be added each time we extend it, so I'm -1 on this
patch as it stands.

This new syntax tries to be consistent and simple. It really doesn't try to
implement an alternative configuration file for pg_dump. The code is simple
and can be easily extended.

What are the benefits of supporting multiple formats?

Show quoted text

Regards,
Dean

#62Tom Lane
tgl@sss.pgh.pa.us
In reply to: Pavel Stehule (#61)
Re: proposal: possibility to read dumped table's name from file

Pavel Stehule <pavel.stehule@gmail.com> writes:

st 25. 11. 2020 v 19:25 odesílatel Dean Rasheed <dean.a.rasheed@gmail.com>
napsal:

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

But I don't understand why? What is a use case? What is a benefit against
command line, or libpq variables? And why should config files be better as
a solution for limited length of command line, when I need to dump
thousands of tables exactly specified?

Because next week somebody will want to dump thousands of functions
selected by name, or schemas selected by name, etc etc. I agree with
the position that we don't want a single-purpose solution. The idea
that the syntax should match the command line switch syntax seems
reasonable, though I'm not wedded to it. (One thing to consider is
how painful will it be for people to quote table names containing
funny characters, for instance. On the command line, we largely
depend on the shell's quoting behavior to solve that, but we'd not
have that infrastructure when reading from a file.)

What are the benefits of supporting multiple formats?

Yeah, that part of Dean's sketch seemed like overkill to me too.

I wasn't very excited about multiple switch files either, though
depending on how the implementation is done, that could be simple
enough to be in the might-as-well category.

One other point that I'm wondering about is that there's really no
value in doing anything here until you get to some thousands of
table names; as long as the list fits in the shell's command line
length limit, you might as well just make a shell script file.
Does pg_dump really have sane performance for that situation, or
are we soon going to be fielding requests to make it not be O(N^2)
in the number of listed tables?

regards, tom lane

#63Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#62)
Re: proposal: possibility to read dumped table's name from file

st 25. 11. 2020 v 21:00 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Pavel Stehule <pavel.stehule@gmail.com> writes:

st 25. 11. 2020 v 19:25 odesílatel Dean Rasheed <

dean.a.rasheed@gmail.com>

napsal:

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

But I don't understand why? What is a use case? What is a benefit against
command line, or libpq variables? And why should config files be better

as

a solution for limited length of command line, when I need to dump
thousands of tables exactly specified?

Because next week somebody will want to dump thousands of functions
selected by name, or schemas selected by name, etc etc. I agree with
the position that we don't want a single-purpose solution. The idea
that the syntax should match the command line switch syntax seems
reasonable, though I'm not wedded to it. (One thing to consider is
how painful will it be for people to quote table names containing
funny characters, for instance. On the command line, we largely
depend on the shell's quoting behavior to solve that, but we'd not
have that infrastructure when reading from a file.)

This is not a problem with the current patch - and the last version of this
patch supports well obscure names.

There was a requirement for supporting all and future pg_dump options - ok
it can make sense. I have not a problem to use instead a line format

"option argument" or "long-option=argument"

This format - can it be a solution? I'll try to rewrite the parser for this
format.

It is implementable, but this is in collision with Stephen's requirement
for human well readable format designed for handy writing. There are
requests that have no intersection. Well readable format needs a more
complex parser. And machine generating in this format needs more fork -
generating flat file is more simple and more robust than generating JSON or
YAML.

What are the benefits of supporting multiple formats?

Yeah, that part of Dean's sketch seemed like overkill to me too.

I wasn't very excited about multiple switch files either, though
depending on how the implementation is done, that could be simple
enough to be in the might-as-well category.

One other point that I'm wondering about is that there's really no
value in doing anything here until you get to some thousands of
table names; as long as the list fits in the shell's command line
length limit, you might as well just make a shell script file.
Does pg_dump really have sane performance for that situation, or
are we soon going to be fielding requests to make it not be O(N^2)
in the number of listed tables?

Performance is another factor, but the command line limit can be easily
touched when table names have maximum width.

Show quoted text

regards, tom lane

#64Dean Rasheed
dean.a.rasheed@gmail.com
In reply to: Pavel Stehule (#63)
Re: proposal: possibility to read dumped table's name from file

On Thu, 26 Nov 2020 at 06:43, Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 25. 11. 2020 v 21:00 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

(One thing to consider is
how painful will it be for people to quote table names containing
funny characters, for instance. On the command line, we largely
depend on the shell's quoting behavior to solve that, but we'd not
have that infrastructure when reading from a file.)

This is not a problem with the current patch - and the last version of this patch supports well obscure names.

Actually, that raises a different possible benefit of passing options
in an options file -- if the user wants to pass in a table name
pattern, it can be a nuisance if the shell's argument processing does
additional unwanted things like globbing and environment variable
substitutions. Using an options file could provide a handy way to
ensure that any option values are interpreted exactly as written,
without any additional mangling.

There was a requirement for supporting all and future pg_dump options - ok it can make sense. I have not a problem to use instead a line format

"option argument" or "long-option=argument"

This format - can it be a solution? I'll try to rewrite the parser for this format.

Yes, that's the sort of thing I was thinking of, to make the feature
more general-purpose.

It is implementable, but this is in collision with Stephen's requirement for human well readable format designed for handy writing. There are requests that have no intersection. Well readable format needs a more complex parser. And machine generating in this format needs more fork - generating flat file is more simple and more robust than generating JSON or YAML.

To be clear, I wasn't suggesting that this patch implement multiple
formats. Just the "plain" format above. Other formats like YAML might
well be more human-readable, and be able to take advantage of values
that are lists to avoid repeating option names, and they would have
the advantage of being readable and writable by other standard tools,
which might be useful. But I think such things would be for the
future. Maybe no one will ever add support for other formats, or maybe
someone will just write a separate external tool to convert YAML or
JSON to our plain format. I don't know. But supporting all pg_dump
options makes more things possible.

I wasn't very excited about multiple switch files either, though
depending on how the implementation is done, that could be simple
enough to be in the might-as-well category.

That's what I was hoping.

Regards,
Dean

#65Tom Lane
tgl@sss.pgh.pa.us
In reply to: Dean Rasheed (#64)
Re: proposal: possibility to read dumped table's name from file

Dean Rasheed <dean.a.rasheed@gmail.com> writes:

Actually, that raises a different possible benefit of passing options
in an options file -- if the user wants to pass in a table name
pattern, it can be a nuisance if the shell's argument processing does
additional unwanted things like globbing and environment variable
substitutions. Using an options file could provide a handy way to
ensure that any option values are interpreted exactly as written,
without any additional mangling.

Huh? Any format we might devise, or borrow, will have to have some
kind of escaping/quoting convention. The idea that "we don't need
that" tends to lead to very ugly workarounds later.

I do agree that the shell's quoting conventions are pretty messy
and so those aren't the ones we should borrow. We could do a lot
worse than to use some established data format like JSON or YAML.
Given that we already have src/common/jsonapi.c, it seems like
JSON would be the better choice of those two.

regards, tom lane

#66Stephen Frost
sfrost@snowman.net
In reply to: Tom Lane (#65)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Dean Rasheed <dean.a.rasheed@gmail.com> writes:

Actually, that raises a different possible benefit of passing options
in an options file -- if the user wants to pass in a table name
pattern, it can be a nuisance if the shell's argument processing does
additional unwanted things like globbing and environment variable
substitutions. Using an options file could provide a handy way to
ensure that any option values are interpreted exactly as written,
without any additional mangling.

Huh? Any format we might devise, or borrow, will have to have some
kind of escaping/quoting convention. The idea that "we don't need
that" tends to lead to very ugly workarounds later.

Agreed.

I do agree that the shell's quoting conventions are pretty messy
and so those aren't the ones we should borrow. We could do a lot
worse than to use some established data format like JSON or YAML.
Given that we already have src/common/jsonapi.c, it seems like
JSON would be the better choice of those two.

JSON doesn't support comments, something that's really useful to have in
configuration files, so I don't agree that it's a sensible thing to use
in this case. JSON also isn't very forgiving, which is also
unfortunate and makes for a poor choice.

This is why I was suggesting TOML up-thread, which is MIT licensed, has
been around for a number of years, supports comments, has sensible
quoting that's easier to deal with than the shell, and has a C (C99)
implementation. It's also used in quite a few other projects.

In a quick look, I suspect it might also be something that could be used
to replace our existing hand-hacked postgresql.conf parser and give us
the ability to handle things a bit cleaner there too...

Thanks,

Stephen

#67Stephen Frost
sfrost@snowman.net
In reply to: Pavel Stehule (#61)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Pavel Stehule (pavel.stehule@gmail.com) wrote:

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

Nobody sent a real use case for introducing the config file. There was a
discussion about formats, and you introduce other dimensions and
variability.

I'm a bit baffled by this because it seems abundently clear to me that
being able to have a config file for pg_dump would be extremely helpful.
There's no shortage of times that I've had to hack up a shell script and
figure out quoting and set up the right set of options for pg_dump,
resulting in things like:

pg_dump \
--host=myserver.com \
--username=postgres \
--schema=public \
--schema=myschema \
--no-comments \
--no-tablespaces \
--file=somedir \
--format=d \
--jobs=5

which really is pretty grotty. Being able to have a config file that
has proper comments would be much better and we could start to extend to
things like "please export schema A to directory A, schema B to
directory B" and other ways of selecting source and destination, and
imagine if we could validate it too, eg:

pg_dump --config=whatever --dry-run

or --check-config maybe.

This isn't a new concept either- export and import tools for other
databases have similar support, eg: Oracle's imp/exp tool, mysqldump
(see: https://dev.mysql.com/doc/refman/8.0/en/option-files.html which
has a TOML-looking format too), pgloader of course has a config file,
etc. We certainly aren't in novel territory here.

Thanks,

Stephen

#68Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#62)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

st 25. 11. 2020 v 21:00 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Pavel Stehule <pavel.stehule@gmail.com> writes:

st 25. 11. 2020 v 19:25 odesílatel Dean Rasheed <

dean.a.rasheed@gmail.com>

napsal:

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

But I don't understand why? What is a use case? What is a benefit against
command line, or libpq variables? And why should config files be better

as

a solution for limited length of command line, when I need to dump
thousands of tables exactly specified?

Because next week somebody will want to dump thousands of functions
selected by name, or schemas selected by name, etc etc. I agree with
the position that we don't want a single-purpose solution. The idea
that the syntax should match the command line switch syntax seems
reasonable, though I'm not wedded to it. (One thing to consider is
how painful will it be for people to quote table names containing
funny characters, for instance. On the command line, we largely
depend on the shell's quoting behavior to solve that, but we'd not
have that infrastructure when reading from a file.)

What are the benefits of supporting multiple formats?

Yeah, that part of Dean's sketch seemed like overkill to me too.

I wasn't very excited about multiple switch files either, though
depending on how the implementation is done, that could be simple
enough to be in the might-as-well category.

One other point that I'm wondering about is that there's really no
value in doing anything here until you get to some thousands of
table names; as long as the list fits in the shell's command line
length limit, you might as well just make a shell script file.
Does pg_dump really have sane performance for that situation, or
are we soon going to be fielding requests to make it not be O(N^2)
in the number of listed tables?

Here is a fresh implementation. I used the name of a new option -
"options-file". Looks more accurate than "config", but the name can be
changed easily anytime.

Any short or long option can be read from this file in simple format - one
option per line. Arguments inside double quotes can be multi lined. Row
comments started by # and can be used everywhere.

The implementation is very generic - support of new options doesn't require
change of this new part code. The parser can ignore white spaces almost
everywhere, where it has sense.

The option should start with "-" or "--" in the options file too, because
this is necessary for good detection if the option is short or long.

Regards

Pavel

Show quoted text

regards, tom lane

Attachments:

pg_dump-options-file.patchtext/x-patch; charset=US-ASCII; name=pg_dump-options-file.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..efdb53f069 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -956,6 +956,34 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dc1d41dd8d..96acb8a1d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -129,6 +131,17 @@ static const CatalogId nilCatalogId = {0, 0};
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static long rowsPerInsert;
+static int numWorkers = 1;
+static int compressLevel = -1;
+static char *optsfilename = NULL;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
@@ -294,14 +307,226 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_options_from_file(char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+#define OPTIONS_FILE_OPT_NUMBER			12
+
+/*
+ * It assign the values of options to related DumpOption fields or to
+ * some global values. It is called from twice. First, for processing
+ * the command line argumens. Second, for processing an options from
+ * options file.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	char	   *endptr;
+
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			numWorkers = atoi(optargstr);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			compressLevel = atoi(optargstr);
+			if (compressLevel < 0 || compressLevel > 9)
+			{
+				pg_log_error("compression level must be in range 0..9");
+				return false;
+			}
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			extra_float_digits = atoi(optargstr);
+			if (extra_float_digits < -15 || extra_float_digits > 3)
+			{
+				pg_log_error("extra_float_digits must be in range -15..3");
+				return false;
+			}
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
 
+		case 10:			/* rows per insert */
+			errno = 0;
+			rowsPerInsert = strtol(optargstr, &endptr, 10);
+
+			if (endptr == optargstr || *endptr != '\0' ||
+				rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+				errno == ERANGE)
+			{
+				pg_log_error("rows-per-insert must be in range %d..%d",
+							 1, INT_MAX);
+				return false;
+			}
+			dopt->dump_inserts = (int) rowsPerInsert;
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPT_NUMBER:			/* filter implementation */
+			optsfilename = pg_strdup(optargstr);
+			break;
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
+
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -309,20 +534,11 @@ main(int argc, char **argv)
 	DumpableObject *boundaryObjs;
 	int			i;
 	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -387,13 +603,17 @@ main(int argc, char **argv)
 		{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPT_NUMBER},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -422,197 +642,20 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
-		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
-				exit_nicely(1);
-		}
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
 	}
 
+	if (optsfilename)
+		read_options_from_file(optsfilename,
+							   &dopt,
+							   short_options,
+							   long_options,
+							   progname);
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1049,6 +1092,7 @@ help(const char *progname)
 	printf(_("  --no-tablespaces             do not dump tablespace assignments\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18635,3 +18679,380 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Reads an option argument from file. Supports double qoutes
+ * bounded strings. In this case multi lines strings are supported.
+ */
+static void
+read_optarg(FILE *fp,
+			char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			exit_invalid_optfile_format(fp,
+										"option '--%.*s' requires an argument at line %d",
+										optname, optnamelen,
+										line->data,
+										*lineno);
+		else
+			exit_invalid_optfile_format(fp,
+										"option '-%.*s' requires an argument at line %d",
+										optname, optnamelen,
+										line->data,
+										*lineno);
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+					exit_invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+
+				if (ferror(fp))
+					fatal("could not read from file \"%s\": %m", filename);
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+			}
+
+			appendStringInfoChar(optargument, *str);
+
+			if (*str++ == '"')
+			{
+				if (*str == '"')
+					str++;
+				else
+					break;
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white spaces */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+		exit_invalid_optfile_format(fp,
+									"unexpected characters after an option's argument at line %d",
+									NULL, 0,
+									line->data,
+									*lineno);
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_options_from_file(char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+			exit_invalid_optfile_format(fp,
+										"non option arguments are not allowed in options file at line %d",
+										NULL, 0,
+										line.data,
+										lineno);
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts,
+									  optname,
+									  optnamelen,
+									  &opt,
+									  &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				/*
+				 * Don't allow --options-file inside options file.
+				 * It is simple protection against cycle.
+				 */
+				if (opt == OPTIONS_FILE_OPT_NUMBER)
+					exit_invalid_optfile_format(fp,
+												"option '%.*s' cannot be used in options file at line %d",
+												optname, optnamelen,
+												line.data,
+												lineno);
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					read_optarg(fp,
+								filename,
+								str,
+								&line,
+								&optargument,
+								optname,
+								optnamelen,
+								true,
+								&lineno);
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+						exit_invalid_optfile_format(fp,
+													"option '--%.*s' doesn't allow an argument at line %d",
+													optname, optnamelen,
+													line.data,
+													lineno);
+				}
+			}
+			else
+				exit_invalid_optfile_format(fp,
+											"unrecognized option '--%.*s' at line %d",
+											optname, optnamelen,
+											line.data,
+											lineno);
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, *optname, &has_arg))
+			{
+				if (has_arg)
+				{
+					read_optarg(fp,
+								filename,
+								str,
+								&line,
+								&optargument,
+								optname,
+								optnamelen,
+								false,
+								&lineno);
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+						exit_invalid_optfile_format(fp,
+													"option '-%.*s' doesn't allow an argument at line %d",
+													optname, optnamelen,
+													line.data,
+													lineno);
+				}
+			}
+			else
+				exit_invalid_optfile_format(fp,
+											"invalid option '-%.*s' at line %d",
+											optname, optnamelen,
+											line.data,
+											lineno);
+
+			opt = *optname;
+		}
+
+		if (opt != 0 &&
+			!process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			exit_nicely(-1);
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_optsfile.pl b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
new file mode 100644
index 0000000000..0955e28a85
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
@@ -0,0 +1,165 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 30;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one #comment\n";
+print $inputfile "-t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "--exclude-table-data=table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-T table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-N public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "--include-foreign-data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: non option arguments are not allowed in options file at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-t";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-t' requires an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-a someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-a' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-r";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-r' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--doesnt-exists";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: unrecognized option '--doesnt-exists' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--data-only badparameter";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--data-only' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--table' requires an argument at line 1/,
+	"broken format check");
#69Pavel Stehule
pavel.stehule@gmail.com
In reply to: Stephen Frost (#67)
Re: proposal: possibility to read dumped table's name from file

pá 27. 11. 2020 v 19:45 odesílatel Stephen Frost <sfrost@snowman.net>
napsal:

Greetings,

* Pavel Stehule (pavel.stehule@gmail.com) wrote:

I agree that being able to configure pg_dump via a config file would
be very useful, but the syntax proposed here feels much more like a
hacked-up syntax designed to meet this one use case, rather than a
good general-purpose design that can be easily extended.

Nobody sent a real use case for introducing the config file. There was a
discussion about formats, and you introduce other dimensions and
variability.

I'm a bit baffled by this because it seems abundently clear to me that
being able to have a config file for pg_dump would be extremely helpful.
There's no shortage of times that I've had to hack up a shell script and
figure out quoting and set up the right set of options for pg_dump,
resulting in things like:

pg_dump \
--host=myserver.com \
--username=postgres \
--schema=public \
--schema=myschema \
--no-comments \
--no-tablespaces \
--file=somedir \
--format=d \
--jobs=5

which really is pretty grotty. Being able to have a config file that
has proper comments would be much better and we could start to extend to
things like "please export schema A to directory A, schema B to
directory B" and other ways of selecting source and destination, and
imagine if we could validate it too, eg:

pg_dump --config=whatever --dry-run

or --check-config maybe.

This isn't a new concept either- export and import tools for other
databases have similar support, eg: Oracle's imp/exp tool, mysqldump
(see: https://dev.mysql.com/doc/refman/8.0/en/option-files.html which
has a TOML-looking format too), pgloader of course has a config file,
etc. We certainly aren't in novel territory here

Still, I am not a fan of this. pg_dump is a simple tool for simple
purposes. It is not a pgloader or any ETL tool. It can be changed in
future, maybe, but still, why? And any time, there will be a question if
pg_dump is a good foundation for massive enhancement in ETL direction. The
development in C is expensive and pg_dump is too Postgres specific, so I
cannot imagine so pg_dump will be used for some complex tasks directly, and
there will be requirements for special configuration. When we have a
pgloader, then we don't need to move pg_dump in the pgloader direction.

Anyway - new patch allows to store any options (one per line) with possible
comments (everywhere in line) and argument's can be across more lines. It
hasn't any more requirements on memory or CPU.

Regards

Pavel

Show quoted text

Thanks,

Stephen

#70Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#68)
4 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On Sat, Nov 28, 2020 at 09:14:35PM +0100, Pavel Stehule wrote:

Any short or long option can be read from this file in simple format - one
option per line. Arguments inside double quotes can be multi lined. Row
comments started by # and can be used everywhere.

Does this support even funkier table names ?

This tests a large number and fraction of characters in dbname/username, so all
of pg_dump has to continue supporting that:
./src/bin/pg_dump/t/010_dump_connstr.pl

I tested and it seems to work with -t "foo�"
But it didn't work with -t "foo\nbar" (literal newline). Fix attached.
If you send another patch, please consider including a test case for quoted
names in long and short options.

+static char *optsfilename = NULL;

+ * It assign the values of options to related DumpOption fields or to
+ * some global values. It is called from twice. First, for processing
+ * the command line argumens. Second, for processing an options from
+ * options file.

This didn't support multiple config files, nor config files which include
config files, as Dean and I mentioned. I think the argument parsers should
themselves call the config file parser, as need be, so the last option
specification should override previous ones.

For example pg_dump --config-file=./pg_dump.conf --blobs should have blobs even
if the config file says --no-blobs. (Command-line arguments normally take
precedence over config files, certainly if the argument is specified "later").
I think it'd be ok if it's recursive. I made a quick hack to do that.

I doubt this will satisfy Stephen. Personally, I would use this if it were a
plain and simple text config file (which for our purposes I would pass on
stdin), and I would almost certainly not use it if it were json. But it'd be
swell if there were a standard config file format, that handled postgresql.conf
and maybe pg_hba.conf.

--
Justin

Attachments:

0001-Re-proposal-possibility-to-read-dumped-table-s-name-.patchtext/x-diff; charset=us-asciiDownload
From d9bac559d235f568a3419070205f05d77853a87f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 28 Nov 2020 15:53:11 -0600
Subject: [PATCH 1/4] Re: proposal: possibility to read dumped table's name
 from file

Pavel Stehule
Nov 28
---
 doc/src/sgml/ref/pg_dump.sgml             |  28 +
 src/bin/pg_dump/pg_dump.c                 | 817 ++++++++++++++++------
 src/bin/pg_dump/t/004_pg_dump_optsfile.pl | 165 +++++
 3 files changed, 812 insertions(+), 198 deletions(-)
 create mode 100644 src/bin/pg_dump/t/004_pg_dump_optsfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..efdb53f069 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -956,6 +956,34 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dc1d41dd8d..96acb8a1d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -129,6 +131,17 @@ static const CatalogId nilCatalogId = {0, 0};
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static long rowsPerInsert;
+static int numWorkers = 1;
+static int compressLevel = -1;
+static char *optsfilename = NULL;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
@@ -294,14 +307,226 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void read_options_from_file(char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+#define OPTIONS_FILE_OPT_NUMBER			12
+
+/*
+ * It assign the values of options to related DumpOption fields or to
+ * some global values. It is called from twice. First, for processing
+ * the command line argumens. Second, for processing an options from
+ * options file.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	char	   *endptr;
+
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			numWorkers = atoi(optargstr);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			compressLevel = atoi(optargstr);
+			if (compressLevel < 0 || compressLevel > 9)
+			{
+				pg_log_error("compression level must be in range 0..9");
+				return false;
+			}
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			extra_float_digits = atoi(optargstr);
+			if (extra_float_digits < -15 || extra_float_digits > 3)
+			{
+				pg_log_error("extra_float_digits must be in range -15..3");
+				return false;
+			}
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
 
+		case 10:			/* rows per insert */
+			errno = 0;
+			rowsPerInsert = strtol(optargstr, &endptr, 10);
+
+			if (endptr == optargstr || *endptr != '\0' ||
+				rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+				errno == ERANGE)
+			{
+				pg_log_error("rows-per-insert must be in range %d..%d",
+							 1, INT_MAX);
+				return false;
+			}
+			dopt->dump_inserts = (int) rowsPerInsert;
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPT_NUMBER:			/* filter implementation */
+			optsfilename = pg_strdup(optargstr);
+			break;
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
+
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -309,20 +534,11 @@ main(int argc, char **argv)
 	DumpableObject *boundaryObjs;
 	int			i;
 	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -387,13 +603,17 @@ main(int argc, char **argv)
 		{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPT_NUMBER},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -422,197 +642,20 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
-		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
-				exit_nicely(1);
-		}
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
 	}
 
+	if (optsfilename)
+		read_options_from_file(optsfilename,
+							   &dopt,
+							   short_options,
+							   long_options,
+							   progname);
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1049,6 +1092,7 @@ help(const char *progname)
 	printf(_("  --no-tablespaces             do not dump tablespace assignments\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18635,3 +18679,380 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Reads an option argument from file. Supports double qoutes
+ * bounded strings. In this case multi lines strings are supported.
+ */
+static void
+read_optarg(FILE *fp,
+			char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			exit_invalid_optfile_format(fp,
+										"option '--%.*s' requires an argument at line %d",
+										optname, optnamelen,
+										line->data,
+										*lineno);
+		else
+			exit_invalid_optfile_format(fp,
+										"option '-%.*s' requires an argument at line %d",
+										optname, optnamelen,
+										line->data,
+										*lineno);
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+					exit_invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+
+				if (ferror(fp))
+					fatal("could not read from file \"%s\": %m", filename);
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+			}
+
+			appendStringInfoChar(optargument, *str);
+
+			if (*str++ == '"')
+			{
+				if (*str == '"')
+					str++;
+				else
+					break;
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white spaces */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+		exit_invalid_optfile_format(fp,
+									"unexpected characters after an option's argument at line %d",
+									NULL, 0,
+									line->data,
+									*lineno);
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_options_from_file(char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+			exit_invalid_optfile_format(fp,
+										"non option arguments are not allowed in options file at line %d",
+										NULL, 0,
+										line.data,
+										lineno);
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts,
+									  optname,
+									  optnamelen,
+									  &opt,
+									  &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				/*
+				 * Don't allow --options-file inside options file.
+				 * It is simple protection against cycle.
+				 */
+				if (opt == OPTIONS_FILE_OPT_NUMBER)
+					exit_invalid_optfile_format(fp,
+												"option '%.*s' cannot be used in options file at line %d",
+												optname, optnamelen,
+												line.data,
+												lineno);
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					read_optarg(fp,
+								filename,
+								str,
+								&line,
+								&optargument,
+								optname,
+								optnamelen,
+								true,
+								&lineno);
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+						exit_invalid_optfile_format(fp,
+													"option '--%.*s' doesn't allow an argument at line %d",
+													optname, optnamelen,
+													line.data,
+													lineno);
+				}
+			}
+			else
+				exit_invalid_optfile_format(fp,
+											"unrecognized option '--%.*s' at line %d",
+											optname, optnamelen,
+											line.data,
+											lineno);
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, *optname, &has_arg))
+			{
+				if (has_arg)
+				{
+					read_optarg(fp,
+								filename,
+								str,
+								&line,
+								&optargument,
+								optname,
+								optnamelen,
+								false,
+								&lineno);
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+						exit_invalid_optfile_format(fp,
+													"option '-%.*s' doesn't allow an argument at line %d",
+													optname, optnamelen,
+													line.data,
+													lineno);
+				}
+			}
+			else
+				exit_invalid_optfile_format(fp,
+											"invalid option '-%.*s' at line %d",
+											optname, optnamelen,
+											line.data,
+											lineno);
+
+			opt = *optname;
+		}
+
+		if (opt != 0 &&
+			!process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			exit_nicely(-1);
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_optsfile.pl b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
new file mode 100644
index 0000000000..0955e28a85
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
@@ -0,0 +1,165 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 30;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one #comment\n";
+print $inputfile "-t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "--exclude-table-data=table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-T table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-N public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "--include-foreign-data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: non option arguments are not allowed in options file at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-t";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-t' requires an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-a someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-a' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-r";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-r' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--doesnt-exists";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: unrecognized option '--doesnt-exists' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--data-only badparameter";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--data-only' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--table' requires an argument at line 1/,
+	"broken format check");
-- 
2.17.0

0002-Fix-short-options-with-double-quotes.patchtext/x-diff; charset=us-asciiDownload
From ca431543d51e0630ba5b8f846ab23888ec7dc499 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 28 Nov 2020 17:00:40 -0600
Subject: [PATCH 2/4] Fix short options with double quotes

---
 src/bin/pg_dump/pg_dump.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 96acb8a1d5..990f890ff4 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -19000,6 +19000,7 @@ read_options_from_file(char *filename,
 			/* process short option */
 			optname = str++;
 			optnamelen = 1;
+			opt = *optname;
 
 			/* skip optional spaces */
 			while (isspace(*str))
@@ -19035,8 +19036,6 @@ read_options_from_file(char *filename,
 											optname, optnamelen,
 											line.data,
 											lineno);
-
-			opt = *optname;
 		}
 
 		if (opt != 0 &&
-- 
2.17.0

0003-typos.patchtext/x-diff; charset=us-asciiDownload
From 98aff704e98e9228dc1e3e04e840785792611537 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 28 Nov 2020 16:58:59 -0600
Subject: [PATCH 3/4] typos

---
 src/bin/pg_dump/pg_dump.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 990f890ff4..43e3a022d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -316,9 +316,9 @@ static void read_options_from_file(char *filename,
 #define OPTIONS_FILE_OPT_NUMBER			12
 
 /*
- * It assign the values of options to related DumpOption fields or to
- * some global values. It is called from twice. First, for processing
- * the command line argumens. Second, for processing an options from
+ * It assigns the values of options to related DumpOption fields or to
+ * some global values. It is called twice. First, for processing
+ * the command line arguments. Second, for processing options from
  * options file.
  */
 static bool
@@ -18711,8 +18711,8 @@ exit_invalid_optfile_format(FILE *fp,
 }
 
 /*
- * Reads an option argument from file. Supports double qoutes
- * bounded strings. In this case multi lines strings are supported.
+ * Reads an option argument from file. Supports double-quoted
+ * strings. In this case multi-line strings are supported.
  */
 static void
 read_optarg(FILE *fp,
@@ -18796,7 +18796,7 @@ read_optarg(FILE *fp,
 		}
 	}
 
-	/* check garbage after optarg, but ignore white spaces */
+	/* check garbage after optarg, but ignore white-space */
 	while (isspace(*str))
 		str++;
 
-- 
2.17.0

0004-Allow-option-file-to-include-options-files.patchtext/x-diff; charset=us-asciiDownload
From e569750c4ce6c6f13f601d2badf54d8df1b83fa5 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 28 Nov 2020 16:59:30 -0600
Subject: [PATCH 4/4] Allow option file to include options files

FIXME: dopt is global and doesn't need to be passed around
---
 src/bin/pg_dump/pg_dump.c | 183 ++++++++++++++++++--------------------
 1 file changed, 86 insertions(+), 97 deletions(-)

diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 43e3a022d5..be6ba5d428 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -140,7 +140,6 @@ static char *use_role = NULL;
 static long rowsPerInsert;
 static int numWorkers = 1;
 static int compressLevel = -1;
-static char *optsfilename = NULL;
 
 /*
  * The default number of rows per INSERT when
@@ -308,13 +307,86 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
 static void read_options_from_file(char *filename,
-								   DumpOptions *dopt,
 								   const char *optstring,
 								   const struct option *longopts,
 								   const char *progname);
 
 #define OPTIONS_FILE_OPT_NUMBER			12
 
+static DumpOptions dopt;
+static struct option long_options[] = {
+	{"data-only", no_argument, NULL, 'a'},
+	{"blobs", no_argument, NULL, 'b'},
+	{"no-blobs", no_argument, NULL, 'B'},
+	{"clean", no_argument, NULL, 'c'},
+	{"create", no_argument, NULL, 'C'},
+	{"dbname", required_argument, NULL, 'd'},
+	{"file", required_argument, NULL, 'f'},
+	{"format", required_argument, NULL, 'F'},
+	{"host", required_argument, NULL, 'h'},
+	{"jobs", 1, NULL, 'j'},
+	{"no-reconnect", no_argument, NULL, 'R'},
+	{"no-owner", no_argument, NULL, 'O'},
+	{"port", required_argument, NULL, 'p'},
+	{"schema", required_argument, NULL, 'n'},
+	{"exclude-schema", required_argument, NULL, 'N'},
+	{"schema-only", no_argument, NULL, 's'},
+	{"superuser", required_argument, NULL, 'S'},
+	{"table", required_argument, NULL, 't'},
+	{"exclude-table", required_argument, NULL, 'T'},
+	{"no-password", no_argument, NULL, 'w'},
+	{"password", no_argument, NULL, 'W'},
+	{"username", required_argument, NULL, 'U'},
+	{"verbose", no_argument, NULL, 'v'},
+	{"no-privileges", no_argument, NULL, 'x'},
+	{"no-acl", no_argument, NULL, 'x'},
+	{"compress", required_argument, NULL, 'Z'},
+	{"encoding", required_argument, NULL, 'E'},
+	{"help", no_argument, NULL, '?'},
+	{"version", no_argument, NULL, 'V'},
+
+	/*
+	 * the following options don't have an equivalent short option letter
+	 */
+	{"attribute-inserts", no_argument, &dopt.column_inserts, 1},
+	{"binary-upgrade", no_argument, &dopt.binary_upgrade, 1},
+	{"column-inserts", no_argument, &dopt.column_inserts, 1},
+	{"disable-dollar-quoting", no_argument, &dopt.disable_dollar_quoting, 1},
+	{"disable-triggers", no_argument, &dopt.disable_triggers, 1},
+	{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
+	{"exclude-table-data", required_argument, NULL, 4},
+	{"extra-float-digits", required_argument, NULL, 8},
+	{"if-exists", no_argument, &dopt.if_exists, 1},
+	{"inserts", no_argument, NULL, 9},
+	{"lock-wait-timeout", required_argument, NULL, 2},
+	{"no-tablespaces", no_argument, &dopt.outputNoTablespaces, 1},
+	{"quote-all-identifiers", no_argument, &quote_all_identifiers, 1},
+	{"load-via-partition-root", no_argument, &dopt.load_via_partition_root, 1},
+	{"role", required_argument, NULL, 3},
+	{"section", required_argument, NULL, 5},
+	{"serializable-deferrable", no_argument, &dopt.serializable_deferrable, 1},
+	{"snapshot", required_argument, NULL, 6},
+	{"strict-names", no_argument, &strict_names, 1},
+	{"use-set-session-authorization", no_argument, &dopt.use_setsessauth, 1},
+	{"no-comments", no_argument, &dopt.no_comments, 1},
+	{"no-publications", no_argument, &dopt.no_publications, 1},
+	{"no-security-labels", no_argument, &dopt.no_security_labels, 1},
+	{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
+	{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
+	{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+	{"no-sync", no_argument, NULL, 7},
+	{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+	{"options-file", required_argument, NULL, OPTIONS_FILE_OPT_NUMBER},
+	{"rows-per-insert", required_argument, NULL, 10},
+	{"include-foreign-data", required_argument, NULL, 11},
+	{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+	{"include-foreign-data-file", required_argument, NULL, 17},
+
+	{NULL, 0, NULL, 0}
+};
+
+const char *short_options = "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 /*
  * It assigns the values of options to related DumpOption fields or to
  * some global values. It is called twice. First, for processing
@@ -328,6 +400,7 @@ process_option(int opt,
 			   const char *progname)
 {
 	char	   *endptr;
+	static int file_depth = 0;
 
 	switch (opt)
 	{
@@ -512,7 +585,16 @@ process_option(int opt,
 			break;
 
 		case OPTIONS_FILE_OPT_NUMBER:			/* filter implementation */
-			optsfilename = pg_strdup(optargstr);
+			if (file_depth++ > 99)
+			{
+				pg_log_error("Don't include so many files including so many files");
+				exit_nicely(1);
+			}
+			read_options_from_file(optargstr,
+					short_options,
+					long_options,
+					progname);
+			file_depth--;
 			break;
 
 		default:
@@ -539,80 +621,6 @@ main(int argc, char **argv)
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-	static DumpOptions dopt;
-
-	static struct option long_options[] = {
-		{"data-only", no_argument, NULL, 'a'},
-		{"blobs", no_argument, NULL, 'b'},
-		{"no-blobs", no_argument, NULL, 'B'},
-		{"clean", no_argument, NULL, 'c'},
-		{"create", no_argument, NULL, 'C'},
-		{"dbname", required_argument, NULL, 'd'},
-		{"file", required_argument, NULL, 'f'},
-		{"format", required_argument, NULL, 'F'},
-		{"host", required_argument, NULL, 'h'},
-		{"jobs", 1, NULL, 'j'},
-		{"no-reconnect", no_argument, NULL, 'R'},
-		{"no-owner", no_argument, NULL, 'O'},
-		{"port", required_argument, NULL, 'p'},
-		{"schema", required_argument, NULL, 'n'},
-		{"exclude-schema", required_argument, NULL, 'N'},
-		{"schema-only", no_argument, NULL, 's'},
-		{"superuser", required_argument, NULL, 'S'},
-		{"table", required_argument, NULL, 't'},
-		{"exclude-table", required_argument, NULL, 'T'},
-		{"no-password", no_argument, NULL, 'w'},
-		{"password", no_argument, NULL, 'W'},
-		{"username", required_argument, NULL, 'U'},
-		{"verbose", no_argument, NULL, 'v'},
-		{"no-privileges", no_argument, NULL, 'x'},
-		{"no-acl", no_argument, NULL, 'x'},
-		{"compress", required_argument, NULL, 'Z'},
-		{"encoding", required_argument, NULL, 'E'},
-		{"help", no_argument, NULL, '?'},
-		{"version", no_argument, NULL, 'V'},
-
-		/*
-		 * the following options don't have an equivalent short option letter
-		 */
-		{"attribute-inserts", no_argument, &dopt.column_inserts, 1},
-		{"binary-upgrade", no_argument, &dopt.binary_upgrade, 1},
-		{"column-inserts", no_argument, &dopt.column_inserts, 1},
-		{"disable-dollar-quoting", no_argument, &dopt.disable_dollar_quoting, 1},
-		{"disable-triggers", no_argument, &dopt.disable_triggers, 1},
-		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
-		{"exclude-table-data", required_argument, NULL, 4},
-		{"extra-float-digits", required_argument, NULL, 8},
-		{"if-exists", no_argument, &dopt.if_exists, 1},
-		{"inserts", no_argument, NULL, 9},
-		{"lock-wait-timeout", required_argument, NULL, 2},
-		{"no-tablespaces", no_argument, &dopt.outputNoTablespaces, 1},
-		{"quote-all-identifiers", no_argument, &quote_all_identifiers, 1},
-		{"load-via-partition-root", no_argument, &dopt.load_via_partition_root, 1},
-		{"role", required_argument, NULL, 3},
-		{"section", required_argument, NULL, 5},
-		{"serializable-deferrable", no_argument, &dopt.serializable_deferrable, 1},
-		{"snapshot", required_argument, NULL, 6},
-		{"strict-names", no_argument, &strict_names, 1},
-		{"use-set-session-authorization", no_argument, &dopt.use_setsessauth, 1},
-		{"no-comments", no_argument, &dopt.no_comments, 1},
-		{"no-publications", no_argument, &dopt.no_publications, 1},
-		{"no-security-labels", no_argument, &dopt.no_security_labels, 1},
-		{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
-		{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
-		{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
-		{"no-sync", no_argument, NULL, 7},
-		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
-		{"options-file", required_argument, NULL, OPTIONS_FILE_OPT_NUMBER},
-		{"rows-per-insert", required_argument, NULL, 10},
-		{"include-foreign-data", required_argument, NULL, 11},
-		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
-		{"include-foreign-data-file", required_argument, NULL, 17},
-
-		{NULL, 0, NULL, 0}
-	};
-
-	const char *short_options = "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
 
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
@@ -649,13 +657,6 @@ main(int argc, char **argv)
 			exit_nicely(1);
 	}
 
-	if (optsfilename)
-		read_options_from_file(optsfilename,
-							   &dopt,
-							   short_options,
-							   long_options,
-							   progname);
-
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -18875,7 +18876,6 @@ is_valid_shortopt(const char *optstring,
  */
 static void
 read_options_from_file(char *filename,
-					   DumpOptions *dopt,
 					   const char *optstring,
 					   const struct option *longopts,
 					   const char *progname)
@@ -18947,17 +18947,6 @@ read_options_from_file(char *filename,
 				while (isspace(*str))
 					str++;
 
-				/*
-				 * Don't allow --options-file inside options file.
-				 * It is simple protection against cycle.
-				 */
-				if (opt == OPTIONS_FILE_OPT_NUMBER)
-					exit_invalid_optfile_format(fp,
-												"option '%.*s' cannot be used in options file at line %d",
-												optname, optnamelen,
-												line.data,
-												lineno);
-
 				if (has_arg)
 				{
 					/* skip optional = */
@@ -19039,7 +19028,7 @@ read_options_from_file(char *filename,
 		}
 
 		if (opt != 0 &&
-			!process_option(opt, optargument.data, dopt, progname))
+			!process_option(opt, optargument.data, &dopt, progname))
 		{
 			fclose(fp);
 			exit_nicely(-1);
-- 
2.17.0

#71Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#70)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

ne 29. 11. 2020 v 0:49 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Sat, Nov 28, 2020 at 09:14:35PM +0100, Pavel Stehule wrote:

Any short or long option can be read from this file in simple format -

one

option per line. Arguments inside double quotes can be multi lined. Row
comments started by # and can be used everywhere.

here is updated patch

Does this support even funkier table names ?

This tests a large number and fraction of characters in dbname/username,
so all
of pg_dump has to continue supporting that:
./src/bin/pg_dump/t/010_dump_connstr.pl

I tested and it seems to work with -t "fooå"
But it didn't work with -t "foo\nbar" (literal newline). Fix attached.
If you send another patch, please consider including a test case for quoted
names in long and short options.

I implemented some basic backslash escaping. I will write more tests, when
there will be good agreement on the main concept.

+static char *optsfilename = NULL;

+ * It assign the values of options to related DumpOption fields or to
+ * some global values. It is called from twice. First, for processing
+ * the command line argumens. Second, for processing an options from
+ * options file.

This didn't support multiple config files, nor config files which include
config files, as Dean and I mentioned. I think the argument parsers should
themselves call the config file parser, as need be, so the last option
specification should override previous ones.

For example pg_dump --config-file=./pg_dump.conf --blobs should have blobs
even
if the config file says --no-blobs. (Command-line arguments normally take
precedence over config files, certainly if the argument is specified
"later").
I think it'd be ok if it's recursive. I made a quick hack to do that.

I did it. I used a different design than you. Making "dopt" be a global
variable looks too invasive. Almost all functions there expect "dopt" as an
argument. But I think it is not necessary.

I implemented two iterations of argument's processing. 1. for options file
(more options-file options are allowed, and nesting is allowed too), 2. all
other arguments from the command line. Any options file is processed only
once - second processing is ignored. So there is no problem with cycles.

The name of the new option - "config-file" or "options-file" ? I prefer
"options-file". "config-file" is valid too, but "options-file" is more
specific, more descriptive (it is self descriptive).

I merged your patch with a fix of typos.

Regards

Pavel

Show quoted text

I doubt this will satisfy Stephen. Personally, I would use this if it
were a
plain and simple text config file (which for our purposes I would pass on
stdin), and I would almost certainly not use it if it were json. But it'd
be
swell if there were a standard config file format, that handled
postgresql.conf
and maybe pg_hba.conf.

--
Justin

Attachments:

pg_dump-options-file-2.patchtext/x-patch; charset=US-ASCII; name=pg_dump-options-file-2.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0aa35cf0c3..1446f377a9 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -956,6 +956,42 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The option <option>--options-file</option> can be used more times,
+        and the nesting is allowed. The options from options files are
+        processed first, other options from command line later. Any option
+        file is processed only one time. In next time the processing is
+        ignored.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dc1d41dd8d..26deebd260 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -123,18 +125,35 @@ static SimpleOidList tabledata_exclude_oids = {NULL, NULL};
 static SimpleStringList foreign_servers_include_patterns = {NULL, NULL};
 static SimpleOidList foreign_servers_include_oids = {NULL, NULL};
 
+static SimpleStringList optsfilenames_processed = {NULL, NULL};
+
 static const CatalogId nilCatalogId = {0, 0};
 
 /* override for standard extra_float_digits setting */
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static long rowsPerInsert;
+static int numWorkers = 1;
+static int compressLevel = -1;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
  */
 #define DUMP_DEFAULT_ROWS_PER_INSERT 1
 
+/*
+ * Option's code of "options-file" option
+ */
+#define OPTIONS_FILE_OPTNUM 12
+
 /*
  * Macro for producing quoted, schema-qualified name of a dumpable object.
  */
@@ -294,14 +313,221 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static bool read_options_from_file(char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+/*
+ * It assigns the values of options to related DumpOption fields or to
+ * some global values. Options file loading is not processed here.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	char	   *endptr;
+
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			numWorkers = atoi(optargstr);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			compressLevel = atoi(optargstr);
+			if (compressLevel < 0 || compressLevel > 9)
+			{
+				pg_log_error("compression level must be in range 0..9");
+				return false;
+			}
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			extra_float_digits = atoi(optargstr);
+			if (extra_float_digits < -15 || extra_float_digits > 3)
+			{
+				pg_log_error("extra_float_digits must be in range -15..3");
+				return false;
+			}
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
+
+		case 10:			/* rows per insert */
+			errno = 0;
+			rowsPerInsert = strtol(optargstr, &endptr, 10);
+
+			if (endptr == optargstr || *endptr != '\0' ||
+				rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+				errno == ERANGE)
+			{
+				pg_log_error("rows-per-insert must be in range %d..%d",
+							 1, INT_MAX);
+				return false;
+			}
+			dopt->dump_inserts = (int) rowsPerInsert;
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPTNUM:	/* reading options file */
+			break;					/* should not be processed here ever */
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
 
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -309,20 +535,11 @@ main(int argc, char **argv)
 	DumpableObject *boundaryObjs;
 	int			i;
 	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -387,13 +604,17 @@ main(int argc, char **argv)
 		{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPTNUM},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -422,197 +643,33 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	/*
+	 * We would to process options-file opts before other options.
+	 * This implements higher priority of options from command line.
+	 * Later processed options wins because overwrite result of
+	 * early processed options.
+	 */
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
+		if (c == OPTIONS_FILE_OPTNUM)
 		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			if (!read_options_from_file(optarg, &dopt, short_options,
+										long_options, progname))
 				exit_nicely(1);
 		}
 	}
 
+	/* reset getopt_long index */
+	optind = 1;
+
+	while ((c = getopt_long(argc, argv, short_options,
+							long_options, &optindex)) != -1)
+	{
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
+	}
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1049,6 +1106,7 @@ help(const char *progname)
 	printf(_("  --no-tablespaces             do not dump tablespace assignments\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18635,3 +18693,450 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and close input file
+ */
+static void
+invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+}
+
+/*
+ * Reads an option argument from file. Supports double-quoted
+ * strings. In this case multi-line strings are supported.
+ */
+static bool
+read_optarg(FILE *fp,
+			char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			invalid_optfile_format(fp,
+								   "option '--%.*s' requires an argument at line %d",
+									optname, optnamelen,
+									line->data,
+									*lineno);
+		else
+			invalid_optfile_format(fp,
+								   "option '-%.*s' requires an argument at line %d",
+								   optname, optnamelen,
+								   line->data,
+								   *lineno);
+		return false;
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			int		c = *str++;
+
+			if (c == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				if (ferror(fp))
+				{
+					pg_log_error("could not read from file \"%s\": %m",
+								 filename);
+					return false;
+				}
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+
+				c = *str++;
+			}
+
+			if (c == '\\')
+			{
+				c = *str++;
+
+				if (c == '\0')
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				switch (c)
+				{
+					case 'n':
+						appendStringInfoChar(optargument, '\n');
+						break;
+
+					case 't':
+						appendStringInfoChar(optargument, '\t');
+						break;
+
+					case '\\':
+						appendStringInfoChar(optargument, '\\');
+						break;
+
+					default:
+						appendStringInfoChar(optargument, '\\');
+						appendStringInfoChar(optargument, c);
+				}
+			}
+			else
+			{
+				appendStringInfoChar(optargument, c);
+
+				if (c == '"')
+				{
+					if (*str == '"')
+						str++;
+					else
+						break;
+				}
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white-space */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+	{
+		invalid_optfile_format(fp,
+							   "unexpected characters after an option's argument at line %d",
+							   NULL, 0,
+							   line->data,
+							   *lineno);
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file. Returns
+ * true, when processing was ok.
+ */
+static bool
+read_options_from_file(char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* Ignore already processed files */
+	if (simple_string_list_member(&optsfilenames_processed,
+								  filename))
+	{
+		pg_log_warning("the options file \"%s\" was processed already, skip this",
+					   filename);
+		return true;
+	}
+
+	simple_string_list_append(&optsfilenames_processed, filename);
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+		{
+			pg_log_error("could not open the input file \"%s\": %m",
+				  filename);
+			return false;
+		}
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+		{
+			invalid_optfile_format(fp,
+								   "non option arguments are not allowed in options file at line %d",
+								   NULL, 0,
+								   line.data,
+								   lineno);
+			return false;
+		}
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts, optname, optnamelen,
+									  &opt, &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, true, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '--%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "unrecognized option '--%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			opt = *optname;
+
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, opt, &has_arg))
+			{
+				if (has_arg)
+				{
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, false, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '-%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "invalid option '-%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+
+		/* nested options file reading */
+		if (opt == OPTIONS_FILE_OPTNUM)
+		{
+			if (!read_options_from_file(optargument.data, dopt, optstring,
+										longopts, progname))
+			{
+				fclose(fp);
+				return false;
+			}
+		}
+		else if (opt != 0 &&
+				 !process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			return false;
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", filename);
+		return false;
+	}
+
+	if (fp != stdin)
+		fclose(fp);
+
+	return true;
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_optsfile.pl b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
new file mode 100644
index 0000000000..0955e28a85
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
@@ -0,0 +1,165 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 30;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one #comment\n";
+print $inputfile "-t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "--exclude-table-data=table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-T table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-N public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "--include-foreign-data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: non option arguments are not allowed in options file at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-t";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-t' requires an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-a someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-a' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-r";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-r' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--doesnt-exists";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: unrecognized option '--doesnt-exists' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--data-only badparameter";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--data-only' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--table' requires an argument at line 1/,
+	"broken format check");
#72Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#71)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

rebase

Regards

Pavel

Attachments:

pg_dump-options-file-3.patchtext/x-patch; charset=US-ASCII; name=pg_dump-options-file-3.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index bcbb7a25fb..f24b3b5262 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -956,6 +956,42 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The option <option>--options-file</option> can be used more times,
+        and the nesting is allowed. The options from options files are
+        processed first, other options from command line later. Any option
+        file is processed only one time. In next time the processing is
+        ignored.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7eb4..17fef1fedf 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -123,18 +125,35 @@ static SimpleOidList tabledata_exclude_oids = {NULL, NULL};
 static SimpleStringList foreign_servers_include_patterns = {NULL, NULL};
 static SimpleOidList foreign_servers_include_oids = {NULL, NULL};
 
+static SimpleStringList optsfilenames_processed = {NULL, NULL};
+
 static const CatalogId nilCatalogId = {0, 0};
 
 /* override for standard extra_float_digits setting */
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static long rowsPerInsert;
+static int numWorkers = 1;
+static int compressLevel = -1;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
  */
 #define DUMP_DEFAULT_ROWS_PER_INSERT 1
 
+/*
+ * Option's code of "options-file" option
+ */
+#define OPTIONS_FILE_OPTNUM 12
+
 /*
  * Macro for producing quoted, schema-qualified name of a dumpable object.
  */
@@ -296,14 +315,221 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static bool read_options_from_file(const char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+/*
+ * It assigns the values of options to related DumpOption fields or to
+ * some global values. Options file loading is not processed here.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	char	   *endptr;
+
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			numWorkers = atoi(optargstr);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			compressLevel = atoi(optargstr);
+			if (compressLevel < 0 || compressLevel > 9)
+			{
+				pg_log_error("compression level must be in range 0..9");
+				return false;
+			}
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			extra_float_digits = atoi(optargstr);
+			if (extra_float_digits < -15 || extra_float_digits > 3)
+			{
+				pg_log_error("extra_float_digits must be in range -15..3");
+				return false;
+			}
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
+
+		case 10:			/* rows per insert */
+			errno = 0;
+			rowsPerInsert = strtol(optargstr, &endptr, 10);
+
+			if (endptr == optargstr || *endptr != '\0' ||
+				rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+				errno == ERANGE)
+			{
+				pg_log_error("rows-per-insert must be in range %d..%d",
+							 1, INT_MAX);
+				return false;
+			}
+			dopt->dump_inserts = (int) rowsPerInsert;
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPTNUM:	/* reading options file */
+			break;					/* should not be processed here ever */
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
 
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -311,20 +537,11 @@ main(int argc, char **argv)
 	DumpableObject *boundaryObjs;
 	int			i;
 	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -389,13 +606,17 @@ main(int argc, char **argv)
 		{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPTNUM},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -424,197 +645,33 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	/*
+	 * We would to process options-file opts before other options.
+	 * This implements higher priority of options from command line.
+	 * Later processed options wins because overwrite result of
+	 * early processed options.
+	 */
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
+		if (c == OPTIONS_FILE_OPTNUM)
 		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			if (!read_options_from_file(optarg, &dopt, short_options,
+										long_options, progname))
 				exit_nicely(1);
 		}
 	}
 
+	/* reset getopt_long index */
+	optind = 1;
+
+	while ((c = getopt_long(argc, argv, short_options,
+							long_options, &optindex)) != -1)
+	{
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
+	}
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1051,6 +1108,7 @@ help(const char *progname)
 	printf(_("  --no-tablespaces             do not dump tablespace assignments\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18779,3 +18837,450 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and close input file
+ */
+static void
+invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+}
+
+/*
+ * Reads an option argument from file. Supports double-quoted
+ * strings. In this case multi-line strings are supported.
+ */
+static bool
+read_optarg(FILE *fp,
+			const char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			invalid_optfile_format(fp,
+								   "option '--%.*s' requires an argument at line %d",
+									optname, optnamelen,
+									line->data,
+									*lineno);
+		else
+			invalid_optfile_format(fp,
+								   "option '-%.*s' requires an argument at line %d",
+								   optname, optnamelen,
+								   line->data,
+								   *lineno);
+		return false;
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			int		c = *str++;
+
+			if (c == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				if (ferror(fp))
+				{
+					pg_log_error("could not read from file \"%s\": %m",
+								 filename);
+					return false;
+				}
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+
+				c = *str++;
+			}
+
+			if (c == '\\')
+			{
+				c = *str++;
+
+				if (c == '\0')
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				switch (c)
+				{
+					case 'n':
+						appendStringInfoChar(optargument, '\n');
+						break;
+
+					case 't':
+						appendStringInfoChar(optargument, '\t');
+						break;
+
+					case '\\':
+						appendStringInfoChar(optargument, '\\');
+						break;
+
+					default:
+						appendStringInfoChar(optargument, '\\');
+						appendStringInfoChar(optargument, c);
+				}
+			}
+			else
+			{
+				appendStringInfoChar(optargument, c);
+
+				if (c == '"')
+				{
+					if (*str == '"')
+						str++;
+					else
+						break;
+				}
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white-space */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+	{
+		invalid_optfile_format(fp,
+							   "unexpected characters after an option's argument at line %d",
+							   NULL, 0,
+							   line->data,
+							   *lineno);
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file. Returns
+ * true, when processing was ok.
+ */
+static bool
+read_options_from_file(const char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* Ignore already processed files */
+	if (simple_string_list_member(&optsfilenames_processed,
+								  filename))
+	{
+		pg_log_warning("the options file \"%s\" was processed already, skip this",
+					   filename);
+		return true;
+	}
+
+	simple_string_list_append(&optsfilenames_processed, filename);
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+		{
+			pg_log_error("could not open the input file \"%s\": %m",
+				  filename);
+			return false;
+		}
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+		{
+			invalid_optfile_format(fp,
+								   "non option arguments are not allowed in options file at line %d",
+								   NULL, 0,
+								   line.data,
+								   lineno);
+			return false;
+		}
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts, optname, optnamelen,
+									  &opt, &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, true, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '--%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "unrecognized option '--%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			opt = *optname;
+
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, opt, &has_arg))
+			{
+				if (has_arg)
+				{
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, false, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '-%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "invalid option '-%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+
+		/* nested options file reading */
+		if (opt == OPTIONS_FILE_OPTNUM)
+		{
+			if (!read_options_from_file(optargument.data, dopt, optstring,
+										longopts, progname))
+			{
+				fclose(fp);
+				return false;
+			}
+		}
+		else if (opt != 0 &&
+				 !process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			return false;
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", filename);
+		return false;
+	}
+
+	if (fp != stdin)
+		fclose(fp);
+
+	return true;
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_optsfile.pl b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
new file mode 100644
index 0000000000..0955e28a85
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_optsfile.pl
@@ -0,0 +1,165 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 30;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = get_new_node('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-t table_one #comment\n";
+print $inputfile "-t table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "--exclude-table-data=table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-T table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "-N public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "--include-foreign-data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: non option arguments are not allowed in options file at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-t";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-t' requires an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-a someforeignserver";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '-a' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "-r";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: invalid option '-r' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--doesnt-exists";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: unrecognized option '--doesnt-exists' at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--data-only badparameter";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--data-only' doesn't allow an argument at line 1/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "--table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--options-file=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: option '--table' requires an argument at line 1/,
+	"broken format check");
#73Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#72)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

út 16. 2. 2021 v 20:32 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

rebase

fresh rebase

Regards

Pavel

Show quoted text

Regards

Pavel

Attachments:

pg_dump-options-file-4.patchtext/x-patch; charset=US-ASCII; name=pg_dump-options-file-4.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 529b167c96..24bfe07ee9 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -1000,6 +1000,42 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The option <option>--options-file</option> can be used more times,
+        and the nesting is allowed. The options from options files are
+        processed first, other options from command line later. Any option
+        file is processed only one time. In next time the processing is
+        ignored.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d0ea489614..665b719c89 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -54,9 +54,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -126,18 +128,35 @@ static SimpleOidList foreign_servers_include_oids = {NULL, NULL};
 static SimpleStringList extension_include_patterns = {NULL, NULL};
 static SimpleOidList extension_include_oids = {NULL, NULL};
 
+static SimpleStringList optsfilenames_processed = {NULL, NULL};
+
 static const CatalogId nilCatalogId = {0, 0};
 
 /* override for standard extra_float_digits setting */
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static long rowsPerInsert;
+static int numWorkers = 1;
+static int compressLevel = -1;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
  */
 #define DUMP_DEFAULT_ROWS_PER_INSERT 1
 
+/*
+ * Option's code of "options-file" option
+ */
+#define OPTIONS_FILE_OPTNUM 12
+
 /*
  * Macro for producing quoted, schema-qualified name of a dumpable object.
  */
@@ -304,14 +323,226 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static bool read_options_from_file(const char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+/*
+ * It assigns the values of options to related DumpOption fields or to
+ * some global values. Options file loading is not processed here.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	char	   *endptr;
+
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'e':			/* include extension(s) */
+			simple_string_list_append(&extension_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			numWorkers = atoi(optargstr);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			compressLevel = atoi(optargstr);
+			if (compressLevel < 0 || compressLevel > 9)
+			{
+				pg_log_error("compression level must be in range 0..9");
+				return false;
+			}
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			extra_float_digits = atoi(optargstr);
+			if (extra_float_digits < -15 || extra_float_digits > 3)
+			{
+				pg_log_error("extra_float_digits must be in range -15..3");
+				return false;
+			}
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
+
+		case 10:			/* rows per insert */
+			errno = 0;
+			rowsPerInsert = strtol(optargstr, &endptr, 10);
+
+			if (endptr == optargstr || *endptr != '\0' ||
+				rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+				errno == ERANGE)
+			{
+				pg_log_error("rows-per-insert must be in range %d..%d",
+							 1, INT_MAX);
+				return false;
+			}
+			dopt->dump_inserts = (int) rowsPerInsert;
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPTNUM:	/* reading options file */
+			break;					/* should not be processed here ever */
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
 
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -319,20 +550,11 @@ main(int argc, char **argv)
 	DumpableObject *boundaryObjs;
 	int			i;
 	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -399,13 +621,17 @@ main(int argc, char **argv)
 		{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPTNUM},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
 		{"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
+		{"include-foreign-data-file", required_argument, NULL, 17},
 
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:e:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -434,202 +660,33 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:e:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	/*
+	 * We would to process options-file opts before other options.
+	 * This implements higher priority of options from command line.
+	 * Later processed options wins because overwrite result of
+	 * early processed options.
+	 */
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
+		if (c == OPTIONS_FILE_OPTNUM)
 		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'e':			/* include extension(s) */
-				simple_string_list_append(&extension_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			if (!read_options_from_file(optarg, &dopt, short_options,
+										long_options, progname))
 				exit_nicely(1);
 		}
 	}
 
+	/* reset getopt_long index */
+	optind = 1;
+
+	while ((c = getopt_long(argc, argv, short_options,
+							long_options, &optindex)) != -1)
+	{
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
+	}
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1082,6 +1139,7 @@ help(const char *progname)
 	printf(_("  --no-toast-compression       do not dump toast compression methods\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18980,3 +19038,450 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and close input file
+ */
+static void
+invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+}
+
+/*
+ * Reads an option argument from file. Supports double-quoted
+ * strings. In this case multi-line strings are supported.
+ */
+static bool
+read_optarg(FILE *fp,
+			const char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			invalid_optfile_format(fp,
+								   "option '--%.*s' requires an argument at line %d",
+									optname, optnamelen,
+									line->data,
+									*lineno);
+		else
+			invalid_optfile_format(fp,
+								   "option '-%.*s' requires an argument at line %d",
+								   optname, optnamelen,
+								   line->data,
+								   *lineno);
+		return false;
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			int		c = *str++;
+
+			if (c == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				if (ferror(fp))
+				{
+					pg_log_error("could not read from file \"%s\": %m",
+								 filename);
+					return false;
+				}
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+
+				c = *str++;
+			}
+
+			if (c == '\\')
+			{
+				c = *str++;
+
+				if (c == '\0')
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				switch (c)
+				{
+					case 'n':
+						appendStringInfoChar(optargument, '\n');
+						break;
+
+					case 't':
+						appendStringInfoChar(optargument, '\t');
+						break;
+
+					case '\\':
+						appendStringInfoChar(optargument, '\\');
+						break;
+
+					default:
+						appendStringInfoChar(optargument, '\\');
+						appendStringInfoChar(optargument, c);
+				}
+			}
+			else
+			{
+				appendStringInfoChar(optargument, c);
+
+				if (c == '"')
+				{
+					if (*str == '"')
+						str++;
+					else
+						break;
+				}
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white-space */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+	{
+		invalid_optfile_format(fp,
+							   "unexpected characters after an option's argument at line %d",
+							   NULL, 0,
+							   line->data,
+							   *lineno);
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file. Returns
+ * true, when processing was ok.
+ */
+static bool
+read_options_from_file(const char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* Ignore already processed files */
+	if (simple_string_list_member(&optsfilenames_processed,
+								  filename))
+	{
+		pg_log_warning("the options file \"%s\" was processed already, skip this",
+					   filename);
+		return true;
+	}
+
+	simple_string_list_append(&optsfilenames_processed, filename);
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+		{
+			pg_log_error("could not open the input file \"%s\": %m",
+				  filename);
+			return false;
+		}
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+		{
+			invalid_optfile_format(fp,
+								   "non option arguments are not allowed in options file at line %d",
+								   NULL, 0,
+								   line.data,
+								   lineno);
+			return false;
+		}
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts, optname, optnamelen,
+									  &opt, &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, true, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '--%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "unrecognized option '--%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			opt = *optname;
+
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, opt, &has_arg))
+			{
+				if (has_arg)
+				{
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, false, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '-%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "invalid option '-%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+
+		/* nested options file reading */
+		if (opt == OPTIONS_FILE_OPTNUM)
+		{
+			if (!read_options_from_file(optargument.data, dopt, optstring,
+										longopts, progname))
+			{
+				fclose(fp);
+				return false;
+			}
+		}
+		else if (opt != 0 &&
+				 !process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			return false;
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", filename);
+		return false;
+	}
+
+	if (fp != stdin)
+		fclose(fp);
+
+	return true;
+}
#74Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#73)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

ne 11. 4. 2021 v 9:48 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

út 16. 2. 2021 v 20:32 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

rebase

rebase

Show quoted text

fresh rebase

Regards

Pavel

Regards

Pavel

Attachments:

pg_dump-options-file-5.patchtext/x-patch; charset=US-ASCII; name=pg_dump-options-file-5.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 67c2cbbec6..742e3515a3 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -1006,6 +1006,42 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The option <option>--options-file</option> can be used more times,
+        and the nesting is allowed. The options from options files are
+        processed first, other options from command line later. Any option
+        file is processed only one time. In next time the processing is
+        ignored.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e384690d94..085791895c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -53,9 +53,11 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -125,18 +127,35 @@ static SimpleOidList foreign_servers_include_oids = {NULL, NULL};
 static SimpleStringList extension_include_patterns = {NULL, NULL};
 static SimpleOidList extension_include_oids = {NULL, NULL};
 
+static SimpleStringList optsfilenames_processed = {NULL, NULL};
+
 static const CatalogId nilCatalogId = {0, 0};
 
 /* override for standard extra_float_digits setting */
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static long rowsPerInsert;
+static int numWorkers = 1;
+static int compressLevel = -1;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
  */
 #define DUMP_DEFAULT_ROWS_PER_INSERT 1
 
+/*
+ * Option's code of "options-file" option
+ */
+#define OPTIONS_FILE_OPTNUM 12
+
 /*
  * Macro for producing quoted, schema-qualified name of a dumpable object.
  */
@@ -300,14 +319,226 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static bool read_options_from_file(const char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+/*
+ * It assigns the values of options to related DumpOption fields or to
+ * some global values. Options file loading is not processed here.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	char	   *endptr;
+
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'e':			/* include extension(s) */
+			simple_string_list_append(&extension_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			numWorkers = atoi(optargstr);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			compressLevel = atoi(optargstr);
+			if (compressLevel < 0 || compressLevel > 9)
+			{
+				pg_log_error("compression level must be in range 0..9");
+				return false;
+			}
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			extra_float_digits = atoi(optargstr);
+			if (extra_float_digits < -15 || extra_float_digits > 3)
+			{
+				pg_log_error("extra_float_digits must be in range -15..3");
+				return false;
+			}
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
+
+		case 10:			/* rows per insert */
+			errno = 0;
+			rowsPerInsert = strtol(optargstr, &endptr, 10);
+
+			if (endptr == optargstr || *endptr != '\0' ||
+				rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
+				errno == ERANGE)
+			{
+				pg_log_error("rows-per-insert must be in range %d..%d",
+							 1, INT_MAX);
+				return false;
+			}
+			dopt->dump_inserts = (int) rowsPerInsert;
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPTNUM:	/* reading options file */
+			break;					/* should not be processed here ever */
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
 
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -315,20 +546,11 @@ main(int argc, char **argv)
 	DumpableObject *boundaryObjs;
 	int			i;
 	int			optindex;
-	char	   *endptr;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	long		rowsPerInsert;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -395,12 +617,15 @@ main(int argc, char **argv)
 		{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPTNUM},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
-
+		{"include-foreign-data-file", required_argument, NULL, 17},
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:e:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -429,202 +654,33 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:e:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	/*
+	 * We would to process options-file opts before other options.
+	 * This implements higher priority of options from command line.
+	 * Later processed options wins because overwrite result of
+	 * early processed options.
+	 */
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
+		if (c == OPTIONS_FILE_OPTNUM)
 		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'e':			/* include extension(s) */
-				simple_string_list_append(&extension_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				numWorkers = atoi(optarg);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				compressLevel = atoi(optarg);
-				if (compressLevel < 0 || compressLevel > 9)
-				{
-					pg_log_error("compression level must be in range 0..9");
-					exit_nicely(1);
-				}
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				extra_float_digits = atoi(optarg);
-				if (extra_float_digits < -15 || extra_float_digits > 3)
-				{
-					pg_log_error("extra_float_digits must be in range -15..3");
-					exit_nicely(1);
-				}
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				errno = 0;
-				rowsPerInsert = strtol(optarg, &endptr, 10);
-
-				if (endptr == optarg || *endptr != '\0' ||
-					rowsPerInsert <= 0 || rowsPerInsert > INT_MAX ||
-					errno == ERANGE)
-				{
-					pg_log_error("rows-per-insert must be in range %d..%d",
-								 1, INT_MAX);
-					exit_nicely(1);
-				}
-				dopt.dump_inserts = (int) rowsPerInsert;
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			if (!read_options_from_file(optarg, &dopt, short_options,
+										long_options, progname))
 				exit_nicely(1);
 		}
 	}
 
+	/* reset getopt_long index */
+	optind = 1;
+
+	while ((c = getopt_long(argc, argv, short_options,
+							long_options, &optindex)) != -1)
+	{
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
+	}
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1073,6 +1129,7 @@ help(const char *progname)
 	printf(_("  --no-toast-compression       do not dump toast compression methods\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18832,3 +18889,450 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and close input file
+ */
+static void
+invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+}
+
+/*
+ * Reads an option argument from file. Supports double-quoted
+ * strings. In this case multi-line strings are supported.
+ */
+static bool
+read_optarg(FILE *fp,
+			const char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			invalid_optfile_format(fp,
+								   "option '--%.*s' requires an argument at line %d",
+									optname, optnamelen,
+									line->data,
+									*lineno);
+		else
+			invalid_optfile_format(fp,
+								   "option '-%.*s' requires an argument at line %d",
+								   optname, optnamelen,
+								   line->data,
+								   *lineno);
+		return false;
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			int		c = *str++;
+
+			if (c == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				if (ferror(fp))
+				{
+					pg_log_error("could not read from file \"%s\": %m",
+								 filename);
+					return false;
+				}
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+
+				c = *str++;
+			}
+
+			if (c == '\\')
+			{
+				c = *str++;
+
+				if (c == '\0')
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				switch (c)
+				{
+					case 'n':
+						appendStringInfoChar(optargument, '\n');
+						break;
+
+					case 't':
+						appendStringInfoChar(optargument, '\t');
+						break;
+
+					case '\\':
+						appendStringInfoChar(optargument, '\\');
+						break;
+
+					default:
+						appendStringInfoChar(optargument, '\\');
+						appendStringInfoChar(optargument, c);
+				}
+			}
+			else
+			{
+				appendStringInfoChar(optargument, c);
+
+				if (c == '"')
+				{
+					if (*str == '"')
+						str++;
+					else
+						break;
+				}
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white-space */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+	{
+		invalid_optfile_format(fp,
+							   "unexpected characters after an option's argument at line %d",
+							   NULL, 0,
+							   line->data,
+							   *lineno);
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file. Returns
+ * true, when processing was ok.
+ */
+static bool
+read_options_from_file(const char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* Ignore already processed files */
+	if (simple_string_list_member(&optsfilenames_processed,
+								  filename))
+	{
+		pg_log_warning("the options file \"%s\" was processed already, skip this",
+					   filename);
+		return true;
+	}
+
+	simple_string_list_append(&optsfilenames_processed, filename);
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+		{
+			pg_log_error("could not open the input file \"%s\": %m",
+				  filename);
+			return false;
+		}
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+		{
+			invalid_optfile_format(fp,
+								   "non option arguments are not allowed in options file at line %d",
+								   NULL, 0,
+								   line.data,
+								   lineno);
+			return false;
+		}
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts, optname, optnamelen,
+									  &opt, &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, true, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '--%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "unrecognized option '--%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			opt = *optname;
+
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, opt, &has_arg))
+			{
+				if (has_arg)
+				{
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, false, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '-%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "invalid option '-%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+
+		/* nested options file reading */
+		if (opt == OPTIONS_FILE_OPTNUM)
+		{
+			if (!read_options_from_file(optargument.data, dopt, optstring,
+										longopts, progname))
+			{
+				fclose(fp);
+				return false;
+			}
+		}
+		else if (opt != 0 &&
+				 !process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			return false;
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", filename);
+		return false;
+	}
+
+	if (fp != stdin)
+		fclose(fp);
+
+	return true;
+}
#75Tomas Vondra
tomas.vondra@enterprisedb.com
In reply to: Pavel Stehule (#74)
Re: proposal: possibility to read dumped table's name from file

Hi,

I started looking at the patch allowing to export just functions [1]https://commitfest.postgresql.org/33/3051/,
and I got pointed to this patch as an alternative approach (to adding a
separate filtering option for every possible object type).

I'm familiar with the customer that inspired Pavel to start working on
this, so I understand the use case he's trying to address - a flexible
way to filter (include/exclude) large number of objects.

IMHO it's a mistake to try to broaden the scope of the patch and require
implementing some universal pg_dump config file, particularly if it
requires "complex" structure or formats like JSON, TOML or whatever.
Maybe that's worth doing, but in my mind it's orthogonal to what this
patch aims (or aimed) to do - filtering objects using rules in a file,
not on the command line.

I believe it's much closer to .gitignore or rsync --filter than to a
full config file. Even if we end up implementing the pg_dump config
file, it'd be nice to keep the filter rules in a separate file and just
reference that file from the config file.

That also means I find it pointless to use an "advanced" format like
JSON or TOML - I think the format should be as simple as possible. Yes,
it has to support all valid identifiers, comments and so on. But I don't
quite see a point in using JSON or similar "full" format. If a simple
format is good enough for rsync or gitignore, why should we insist on
using something more complex?

OTOH I don't quite like the current approach of simply reading options
from a file, because that requires adding new command-line options for
each type of object we want to support. Which seems to contradict the
idea of "general filter" method as mentioned in [1]https://commitfest.postgresql.org/33/3051/.

So if it was up to me, I'd go back to the original format or something
close it. So something like this:

[+-] OBJECT_TYPE_PATTERN OBJECT_NAME_PATTERN

regards

[1]: https://commitfest.postgresql.org/33/3051/

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#76Daniel Gustafsson
daniel@yesql.se
In reply to: Tomas Vondra (#75)
Re: proposal: possibility to read dumped table's name from file

On 10 Jul 2021, at 17:47, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:

So if it was up to me, I'd go back to the original format or something close it. So something like this:

[+-] OBJECT_TYPE_PATTERN OBJECT_NAME_PATTERN

That still leaves the parsing with quoting and escaping that needs to be done
less trivial and more bespoke than what meets the eye, no?

As mentioned upthread, I'm still hesitant to add a file format which deosn't
have any version information of sorts for distinguishing it from when the
inevitable "now wouldn't it be nice if we could do this too" patch which we all
know will come. The amount of selectivity switches we have for pg_dump is an
indication about just how much control users like, this will no doubt be
subject to the same.

--
Daniel Gustafsson https://vmware.com/

#77Tomas Vondra
tomas.vondra@enterprisedb.com
In reply to: Daniel Gustafsson (#76)
Re: proposal: possibility to read dumped table's name from file

On 7/13/21 12:08 AM, Daniel Gustafsson wrote:

On 10 Jul 2021, at 17:47, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:

So if it was up to me, I'd go back to the original format or something close it. So something like this:

[+-] OBJECT_TYPE_PATTERN OBJECT_NAME_PATTERN

That still leaves the parsing with quoting and escaping that needs to be done
less trivial and more bespoke than what meets the eye, no?

Yes, it'd require proper escaping/quoting of the fields/identifiers etc.

As mentioned upthread, I'm still hesitant to add a file format which deosn't
have any version information of sorts for distinguishing it from when the
inevitable "now wouldn't it be nice if we could do this too" patch which we all
know will come. The amount of selectivity switches we have for pg_dump is an
indication about just how much control users like, this will no doubt be
subject to the same.

I'm not going to fight against some sort of versioning, but I think
keeping the scope as narrow as possible would make it unnecessary. That
is, let's stick to the original goal to allow passing filtering rules
that would not fit on the command-line, and maybe let's make it a bit
more flexible to support other object types etc.

IMHO the filtering rules are simple enough to not really need elaborate
versioning, and if a more advanced rule is proposed in the future it can
be supported in the existing format (extra field, ...).

Of course, maybe my imagination is not wild enough.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#78Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Tomas Vondra (#77)
Re: proposal: possibility to read dumped table's name from file

On 2021-Jul-13, Tomas Vondra wrote:

I'm not going to fight against some sort of versioning, but I think keeping
the scope as narrow as possible would make it unnecessary. That is, let's
stick to the original goal to allow passing filtering rules that would not
fit on the command-line, and maybe let's make it a bit more flexible to
support other object types etc.

IMHO the filtering rules are simple enough to not really need elaborate
versioning, and if a more advanced rule is proposed in the future it can be
supported in the existing format (extra field, ...).

I don't understand why is versioning needed for this file. Surely we
can just define some line-based grammar that's accepted by the current
pg_dump[1]your proposal of "[+-] OBJTYPE OBJIDENT" plus empty lines allowed plus lines starting with # are comments, seems plenty. Any line not following that format would cause an error to be thrown. and that would satisfy the current need as well as allowing
for extending the grammar in the future; even JSON or Windows-INI format
(ugh?) if that's necessary to tailor the output file in some other way
not covered by that.

[1]: your proposal of "[+-] OBJTYPE OBJIDENT" plus empty lines allowed plus lines starting with # are comments, seems plenty. Any line not following that format would cause an error to be thrown.
plus lines starting with # are comments, seems plenty. Any line not
following that format would cause an error to be thrown.

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

#79Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#78)
Re: proposal: possibility to read dumped table's name from file

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

[1] your proposal of "[+-] OBJTYPE OBJIDENT" plus empty lines allowed
plus lines starting with # are comments, seems plenty. Any line not
following that format would cause an error to be thrown.

I'd like to see some kind of keyword on each line, so that we could extend
the command set by adding new keywords. As this stands, I fear we'd end
up using random punctuation characters in place of [+-], which seems
pretty horrid from a readability standpoint.

I think that this file format should be designed with an eye to allowing
every, or at least most, pg_dump options to be written in the file rather
than on the command line. I don't say we have to *implement* that right
now; but if the format spec is incapable of being extended to meet
requests like that one, I think we'll regret it. This line of thought
suggests that the initial commands ought to match the existing
include/exclude switches, at least approximately.

Hence I suggest

include table PATTERN
exclude table PATTERN

which ends up being the above but with words not [+-].

regards, tom lane

#80Stephen Frost
sfrost@snowman.net
In reply to: Tom Lane (#79)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

[1] your proposal of "[+-] OBJTYPE OBJIDENT" plus empty lines allowed
plus lines starting with # are comments, seems plenty. Any line not
following that format would cause an error to be thrown.

I'd like to see some kind of keyword on each line, so that we could extend
the command set by adding new keywords. As this stands, I fear we'd end
up using random punctuation characters in place of [+-], which seems
pretty horrid from a readability standpoint.

I agree that it'd end up being bad with single characters.

I think that this file format should be designed with an eye to allowing
every, or at least most, pg_dump options to be written in the file rather
than on the command line. I don't say we have to *implement* that right
now; but if the format spec is incapable of being extended to meet
requests like that one, I think we'll regret it. This line of thought
suggests that the initial commands ought to match the existing
include/exclude switches, at least approximately.

I agree that we want to have an actual config file that allows just
about every pg_dump option. I'm also fine with saying that we don't
have to implement that initially but the format should be one which can
be extended to allow that.

Hence I suggest

include table PATTERN
exclude table PATTERN

which ends up being the above but with words not [+-].

Which ends up inventing yet-another-file-format which people will end up
writing generators and parsers for. Which is exactly what I was arguing
we really should be trying to avoid doing.

I definitely feel that we should have a way to allow anything that can
be created as an object in the database to be explicitly included in the
file and that means whatever we do need to be able to handle objects
that have names that span multiple lines, etc. It's not clear how the
above would. As I recall, the proposed patch didn't have anything for
handling that, which was one of the issues I had with it and is why I
bring it up again.

Thanks,

Stephen

#81Tomas Vondra
tomas.vondra@enterprisedb.com
In reply to: Stephen Frost (#80)
Re: proposal: possibility to read dumped table's name from file

On 7/13/21 3:40 PM, Stephen Frost wrote:

Greetings,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

[1] your proposal of "[+-] OBJTYPE OBJIDENT" plus empty lines allowed
plus lines starting with # are comments, seems plenty. Any line not
following that format would cause an error to be thrown.

I'd like to see some kind of keyword on each line, so that we could extend
the command set by adding new keywords. As this stands, I fear we'd end
up using random punctuation characters in place of [+-], which seems
pretty horrid from a readability standpoint.

I agree that it'd end up being bad with single characters.

The [+-] format is based on what rsync does, so there's at least some
precedent for that, and IMHO it's fairly readable. I agree the rest of
the rule (object type, ...) may be a bit more verbose.

I think that this file format should be designed with an eye to allowing
every, or at least most, pg_dump options to be written in the file rather
than on the command line. I don't say we have to *implement* that right
now; but if the format spec is incapable of being extended to meet
requests like that one, I think we'll regret it. This line of thought
suggests that the initial commands ought to match the existing
include/exclude switches, at least approximately.

I agree that we want to have an actual config file that allows just
about every pg_dump option. I'm also fine with saying that we don't
have to implement that initially but the format should be one which can
be extended to allow that.

I understand the desire to have a config file that may contain all
pg_dump options, but I really don't see why we'd want to mix that with
the file containing filter rules.

I think those should be separate, one of the reasons being that I find
it desirable to be able to "include" the filter rules into different
pg_dump configs. That also means the format for the filter rules can be
much simpler.

It's also not clear to me whether the single-file approach would allow
filtering not supported by actual pg_dump option, for example.

Hence I suggest

include table PATTERN
exclude table PATTERN

which ends up being the above but with words not [+-].

Work for me.

Which ends up inventing yet-another-file-format which people will end up
writing generators and parsers for. Which is exactly what I was arguing
we really should be trying to avoid doing.

People will have to write generators *in any case* because how else
would you use this? Unless we also provide tools to manipulate that file
(which seems rather futile), they'll have to do that. Even if we used
JSON/YAML/TOML/... they'd still need to deal with the semantics of the
file format.

FWIW I don't understand why would they need to write parsers. That's
something we'd need to do to process the file. I think the case when the
filter file needs to be modified is rather rare - it certainly is not
what the original use case Pavel tried to address needs. (I know that
customer and the filter would be generated and used for a single dump.)

My opinion is that the best solution (to make both generators and
parsers simple) is to keep the format itself as simple as possible.
Which is exactly why I'm arguing for only addressing the filtering, not
trying to invent a "universal" pg_dump config file format.

I definitely feel that we should have a way to allow anything that can
be created as an object in the database to be explicitly included in the
file and that means whatever we do need to be able to handle objects
that have names that span multiple lines, etc. It's not clear how the
above would. As I recall, the proposed patch didn't have anything for
handling that, which was one of the issues I had with it and is why I
bring it up again.

I really don't understand why you think the current format can't do
escaping/quoting or handle names spanning multiple lines. The fact that
the original patch did not handle that correctly is a bug, but it does
not mean the format can't handle that.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#82Daniel Gustafsson
daniel@yesql.se
In reply to: Tomas Vondra (#81)
Re: proposal: possibility to read dumped table's name from file

On 13 Jul 2021, at 18:14, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:

FWIW I don't understand why would they need to write parsers.

It's quite common to write unit tests for VM recipes/playbooks wheen using
tools like Chef etc, parsing and checking the installed/generated files is part
of that. This would be one very real use case for writing a parser.

I think the case when the filter file needs to be modified is rather rare - it certainly is not what the original use case Pavel tried to address needs. (I know that customer and the filter would be generated and used for a single dump.)

I'm not convinced that basing design decisions on a single customer reference
who only want to use the code once is helpful. I hear what you're saying, but
I think this will see more diverse use cases than what we can foresee here.

--
Daniel Gustafsson https://vmware.com/

#83Daniel Gustafsson
daniel@yesql.se
In reply to: Alvaro Herrera (#78)
Re: proposal: possibility to read dumped table's name from file

On 13 Jul 2021, at 00:59, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

On 2021-Jul-13, Tomas Vondra wrote:

I'm not going to fight against some sort of versioning, but I think keeping
the scope as narrow as possible would make it unnecessary. That is, let's
stick to the original goal to allow passing filtering rules that would not
fit on the command-line, and maybe let's make it a bit more flexible to
support other object types etc.

IMHO the filtering rules are simple enough to not really need elaborate
versioning, and if a more advanced rule is proposed in the future it can be
supported in the existing format (extra field, ...).

I don't understand why is versioning needed for this file. Surely we
can just define some line-based grammar that's accepted by the current
pg_dump[1] and that would satisfy the current need as well as allowing
for extending the grammar in the future; even JSON or Windows-INI format
(ugh?) if that's necessary to tailor the output file in some other way
not covered by that.

I wasn't expressing myself very well; by "versioning" I mean a way to be able
to add to/change/fix the format and still be able to deterministically parse it
without having to resort to ugly heuristics and hacks. If that's achieved by
an explicit version number or if it's an inherit characteristic of the format
doesn't really matter (to me). My worry is that the very simple proposed
format might not fit that bill, but since I don't know what the future of the
feature might bring it's (mostly) a gut feeling.

--
Daniel Gustafsson https://vmware.com/

#84Stephen Frost
sfrost@snowman.net
In reply to: Daniel Gustafsson (#82)
Re: proposal: possibility to read dumped table's name from file

Greetings,

On Tue, Jul 13, 2021 at 16:44 Daniel Gustafsson <daniel@yesql.se> wrote:

On 13 Jul 2021, at 18:14, Tomas Vondra <tomas.vondra@enterprisedb.com>

wrote:

FWIW I don't understand why would they need to write parsers.

It's quite common to write unit tests for VM recipes/playbooks wheen using
tools like Chef etc, parsing and checking the installed/generated files is
part
of that. This would be one very real use case for writing a parser.

Consider pgAdmin and the many other tools which essentially embed pg_dump
and pg_restore. There’s no shortage of use cases for a variety of tools to
be able to understand, read, parse, generate, rewrite, and probably do
more, with such a pg_dump/restore config file.

I think the case when the filter file needs to be modified is rather rare
- it certainly is not what the original use case Pavel tried to address
needs. (I know that customer and the filter would be generated and used for
a single dump.)

I'm not convinced that basing design decisions on a single customer
reference
who only want to use the code once is helpful.

Agreed.

Thanks,

Stephen

Show quoted text
#85Tomas Vondra
tomas.vondra@enterprisedb.com
In reply to: Stephen Frost (#84)
Re: proposal: possibility to read dumped table's name from file

On 7/13/21 10:55 PM, Stephen Frost wrote:

Greetings,

On Tue, Jul 13, 2021 at 16:44 Daniel Gustafsson <daniel@yesql.se
<mailto:daniel@yesql.se>> wrote:

On 13 Jul 2021, at 18:14, Tomas Vondra

<tomas.vondra@enterprisedb.com
<mailto:tomas.vondra@enterprisedb.com>> wrote:

FWIW I don't understand why would they need to write parsers.

It's quite common to write unit tests for VM recipes/playbooks wheen
using
tools like Chef etc, parsing and checking the installed/generated
files is part
of that. This would be one very real use case for writing a parser.

Consider pgAdmin and the many other tools which essentially embed
pg_dump and pg_restore.  There’s no shortage of use cases for a variety
of tools to be able to understand, read, parse, generate, rewrite, and
probably do more, with such a pg_dump/restore config file.

Sure. Which is why I'm advocating for the simplest possible format (and
not expanding the scope of this patch beyond filtering), because that
makes this kind of processing simpler.

I think the case when the filter file needs to be modified is

rather rare - it certainly is not what the original use case Pavel
tried to address needs. (I know that customer and the filter would
be generated and used for a single dump.)

I'm not convinced that basing design decisions on a single customer
reference
who only want to use the code once is helpful.

Agreed.

I wasn't really basing this on a single customer - that was merely an
example, of course. FWIW Justin Pryzby already stated having to use some
more complex format would likely mean they would not use the feature, so
that's another data point to consider.

FWIW I believe it's clear what my opinions on this topic are. Repeating
that seems a bit pointless, so I'll step aside and let this thread move
forward in whatever direction.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#86Stephen Frost
sfrost@snowman.net
In reply to: Tomas Vondra (#85)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Tomas Vondra (tomas.vondra@enterprisedb.com) wrote:

On 7/13/21 10:55 PM, Stephen Frost wrote:

On Tue, Jul 13, 2021 at 16:44 Daniel Gustafsson <daniel@yesql.se
<mailto:daniel@yesql.se>> wrote:

On 13 Jul 2021, at 18:14, Tomas Vondra

<tomas.vondra@enterprisedb.com
<mailto:tomas.vondra@enterprisedb.com>> wrote:

FWIW I don't understand why would they need to write parsers.

It's quite common to write unit tests for VM recipes/playbooks wheen
using
tools like Chef etc, parsing and checking the installed/generated
files is part
of that. This would be one very real use case for writing a parser.

Consider pgAdmin and the many other tools which essentially embed pg_dump
and pg_restore.  There’s no shortage of use cases for a variety of tools
to be able to understand, read, parse, generate, rewrite, and probably do
more, with such a pg_dump/restore config file.

Sure. Which is why I'm advocating for the simplest possible format (and not
expanding the scope of this patch beyond filtering), because that makes this
kind of processing simpler.

The simplest possible format isn't going to work with all the different
pg_dump options and it still isn't going to be 'simple' since it needs
to work with the flexibility that we have in what we support for object
names, and is still going to require people write a new parser and
generator for it instead of using something existing.

I don't know that the options that I suggested previously would
definitely work or not but they at least would allow other projects like
pgAdmin to leverage existing code for parsing and generating these
config files. I'm not completely against inventing something new, but
I'd really prefer that we at least try to make something existing work
first before inventing something new that everyone is going to have to
deal with. If we do invent a new thing for $reasons, then we should
really look at what exists today and try to design it properly instead
of just throwing something together and formally document it because
it's absolutely going to become a standard of sorts that people are
going to almost immediately write their own parsers/generators in
various languages for.

Thanks,

Stephen

#87Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Stephen Frost (#86)
Re: proposal: possibility to read dumped table's name from file

On 2021-Jul-13, Stephen Frost wrote:

The simplest possible format isn't going to work with all the different
pg_dump options and it still isn't going to be 'simple' since it needs
to work with the flexibility that we have in what we support for object
names,

That's fine. If people want a mechanism that allows changing the other
pg_dump options that are not related to object filtering, they can
implement a configuration file for that.

and is still going to require people write a new parser and
generator for it instead of using something existing.

Sure. That's not part of this patch.

I don't know that the options that I suggested previously would
definitely work or not but they at least would allow other projects like
pgAdmin to leverage existing code for parsing and generating these
config files.

Keep in mind that this patch is not intended to help pgAdmin
specifically. It would be great if pgAdmin uses the functionality
implemented here, but if they decide not to, that's not terrible. They
have survived decades without a pg_dump configuration file; they still
can.

There are several votes in this thread for pg_dump to gain functionality
to filter objects based on a simple specification -- particularly one
that can be written using shell pipelines. This patch gives it.

I'm not completely against inventing something new, but I'd really
prefer that we at least try to make something existing work first
before inventing something new that everyone is going to have to deal
with.

That was discussed upthread and led nowhere.

--
Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/
"Nunca se desea ardientemente lo que solo se desea por razón" (F. Alexandre)

#88Stephen Frost
sfrost@snowman.net
In reply to: Alvaro Herrera (#87)
Re: proposal: possibility to read dumped table's name from file

Greetings,

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

On 2021-Jul-13, Stephen Frost wrote:

The simplest possible format isn't going to work with all the different
pg_dump options and it still isn't going to be 'simple' since it needs
to work with the flexibility that we have in what we support for object
names,

That's fine. If people want a mechanism that allows changing the other
pg_dump options that are not related to object filtering, they can
implement a configuration file for that.

It's been said multiple times that people *do* want that and that they
want it to all be part of this one file, and specifically that they
don't want to end up with a file structure that actively works against
allowing other options to be added to it.

I don't know that the options that I suggested previously would
definitely work or not but they at least would allow other projects like
pgAdmin to leverage existing code for parsing and generating these
config files.

Keep in mind that this patch is not intended to help pgAdmin
specifically. It would be great if pgAdmin uses the functionality
implemented here, but if they decide not to, that's not terrible. They
have survived decades without a pg_dump configuration file; they still
can.

The adding of a config file for pg_dump should specifically be looking
at pgAdmin as the exact use-case for having such a capability.

There are several votes in this thread for pg_dump to gain functionality
to filter objects based on a simple specification -- particularly one
that can be written using shell pipelines. This patch gives it.

And several votes for having a config file that supports, or at least
can support in the future, the various options which pg_dump supports-
and active voices against having a new file format that doesn't allow
for that.

I'm not completely against inventing something new, but I'd really
prefer that we at least try to make something existing work first
before inventing something new that everyone is going to have to deal
with.

That was discussed upthread and led nowhere.

You're right- no one followed up on that. Instead, one group continues
to push for 'simple' and to just accept what's been proposed, while
another group counters that we should be looking at the broader design
question and work towards a solution which will work for us down the
road, and not just right now.

One thing remains clear- there's no consensus here.

Thanks,

Stephen

#89Pavel Stehule
pavel.stehule@gmail.com
In reply to: Stephen Frost (#88)
Re: proposal: possibility to read dumped table's name from file

Hi

You're right- no one followed up on that. Instead, one group continues
to push for 'simple' and to just accept what's been proposed, while
another group counters that we should be looking at the broader design
question and work towards a solution which will work for us down the
road, and not just right now.

One thing remains clear- there's no consensus here.

I think there should be some misunderstanding about the target of this
patch, and I am afraid so there cannot be consensus, because the people are
speaking about two very different features. And it is not possible to push
it to one thing. It cannot work I am afraid.

1. The main target of this patch is to solve the problem with the too large
command line of pg_dump when there are a lot of dumped objects. You need to
call pg_dump only once to ensure dump in one transaction. And sometimes it
is not possible to use wild characters effectively, because the state of
objects is in different databases. Enhancing the length of the command line
is not secure, and there are other production issues. In this case you need
a very simple format - just because you want to use pg_dump in pipe. This
format should be line oriented - and usually it will contain just "dump
this table, dump second table". Nothing else. Nobody will read this format,
nobody will edit this format. Because the main platform for this format is
probably the UNIX shell, the format should be simple. I really don't see
any joy in generating JSON and parsing JSON later. These data will be
processed locally. This is one purpose designed format, and it is not
designed for holding configuration. For this purpose the complex format has
not any advantage. There is not a problem with parsing JSON or other
formats on the pg_dump side, but it is pretty hard to generate valid JSON
from bash script. For a unix shell we need the most possible simple format.
Theoretically this format (this file) can hold any pg_dump's option, but
for usual streaming processing the only filter's options will be there.
Originally this feature had the name "filter file". There are a lot of
examples of successful filter's file formats in the UNIX world, and I think
so nobody doubts about sense and usability. Probably there is a consensus
so filter's files are not config files.

The format of the filter file can look like "+d tablename" or "include data
tablename". If we find a consensus so the filter file is a good thing, then
the format design and implementation is easy work. Isn't problem to invent
comment lines.

2. Is true, so there is only a small step from filter's file to option's
file. I rewrote this patch in this direction. The advantage is universality
- it can support any options without necessity to modify related code.
Still this format is not difficult for producers, and it is simple for
parsing. Now, the format should be defined by command line format: "-t
tablename" or "--table tablename" or "table tablename". There can be issues
related to different parsers in shell and in implemented code, but it can
be solved. Isn't problem to introduce comment lines. The big advantage is
simplicity of usage, simplicity of implementation - more the implementation
is generic.

3. But the option's file is just a small step to config file. I can imagine
somebody wanting to store typical configuration (and usual options) for
psql, pg_dump, pg_restore, pgAdmin, ... somewhere. The config files are
very different creatures than filter's files. Although they can be
generated, usually are edited and can be very complex. There can be shared
parts for all applications, and specific sections for psql, and specific
sections for every database. The config files can be brutally complex. The
simple text format is not good for this purpose. And some people prefer
YAML, some people hate this format. Other people prefer XML or JSON or
anything else. Sometimes the complexity of config files is too big, and
people prefer startup scripting.

Although there is an intersection between filter's files and config files,
I see very big differences in usage. Filter's files are usually temporal
and generated and non shared. Config file's are persistent, usually
manually modified and can be shared. The requests are different, and should
be different too. I don't propose any configuration's file related
features, and my proposal doesn't block the introduction of configuration's
file in any format in future. I think these features are very different,
and should be implemented differently. The filter's file or option's file
will be a pretty ugly config file, and config's file will be a pretty
impractical filter's file.

So can we talk about implementation of filter's file or option's file? And
can we talk about implementation config's files in separate topics? Without
it, I am afraid so there is no possibility of finding an agreement and
moving forward.

Regards

Pavel

Show quoted text

Thanks,

Stephen

#90Tomas Vondra
tomas.vondra@enterprisedb.com
In reply to: Stephen Frost (#88)
Re: proposal: possibility to read dumped table's name from file

On 7/14/21 2:18 AM, Stephen Frost wrote:

Greetings,

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

On 2021-Jul-13, Stephen Frost wrote:

The simplest possible format isn't going to work with all the different
pg_dump options and it still isn't going to be 'simple' since it needs
to work with the flexibility that we have in what we support for object
names,

That's fine. If people want a mechanism that allows changing the other
pg_dump options that are not related to object filtering, they can
implement a configuration file for that.

It's been said multiple times that people *do* want that and that they
want it to all be part of this one file, and specifically that they
don't want to end up with a file structure that actively works against
allowing other options to be added to it.

I have no problem believing some people want to be able to specify
pg_dump parameters in a file, similarly to IMPDP/EXPDP parameter files
etc. That seems useful, but I doubt they considered the case with many
filter rules ... which is what "my people" want.

Not sure how keeping the filter rules in a separate file (which I assume
is what you mean by "file structure"), with a format tailored for filter
rules, works *actively* against adding options to the "main" config.

I'm not buying the argument that keeping some of the stuff in a separate
file is an issue - plenty of established tools do that, the concept of
"including" a config is not a radical new thing, and I don't expect we'd
have many options supported by a file.

In any case, I think user input is important, but ultimately it's up to
us to reconcile the conflicting requirements coming from various users
and come up with a reasonable compromise design.

I don't know that the options that I suggested previously would
definitely work or not but they at least would allow other projects like
pgAdmin to leverage existing code for parsing and generating these
config files.

Keep in mind that this patch is not intended to help pgAdmin
specifically. It would be great if pgAdmin uses the functionality
implemented here, but if they decide not to, that's not terrible. They
have survived decades without a pg_dump configuration file; they still
can.

The adding of a config file for pg_dump should specifically be looking
at pgAdmin as the exact use-case for having such a capability.

There are several votes in this thread for pg_dump to gain functionality
to filter objects based on a simple specification -- particularly one
that can be written using shell pipelines. This patch gives it.

And several votes for having a config file that supports, or at least
can support in the future, the various options which pg_dump supports-
and active voices against having a new file format that doesn't allow
for that.

IMHO the whole "problem" here stems from the question whether there
should be a single universal pg_dump config file, containing everything
including the filter rules. I'm of the opinion it's better to keep the
filter rules separate, mainly because:

1) simplicity - Options (key/value) and filter rules (with more internal
structure) seem quite different, and mixing them in the same file will
just make the format more complex.

2) flexibility - Keeping the filter rules in a separate file makes it
easier to reuse the same set of rules with different pg_dump configs,
specified in (much smaller) config files.

So in principle, the "main" config could use e.g. TOML or whatever we
find most suitable for this type of key/value config file (or we could
just use the same format as for postgresql.conf et al). And the filter
rules could use something as simple as CSV (yes, I know it's not great,
but there's plenty of parsers, it handles multi-line strings etc.).

I'm not completely against inventing something new, but I'd really
prefer that we at least try to make something existing work first
before inventing something new that everyone is going to have to deal
with.

That was discussed upthread and led nowhere.

You're right- no one followed up on that. Instead, one group continues
to push for 'simple' and to just accept what's been proposed, while
another group counters that we should be looking at the broader design
question and work towards a solution which will work for us down the
road, and not just right now.

I have quite thick skin, but I have to admit I rather dislike how this
paints the people arguing for simplicity.

IMO simplicity is a perfectly legitimate (and desirable) design feature,
and simpler solutions often fare better in the long run. Yes, we need to
look at the broader design, no doubt about that.

One thing remains clear- there's no consensus here.

True.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#91Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#74)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

st 12. 5. 2021 v 8:22 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

ne 11. 4. 2021 v 9:48 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

út 16. 2. 2021 v 20:32 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

rebase

rebase

fresh rebase

fresh rebase

Show quoted text

Regards

Pavel

Regards

Pavel

Attachments:

pg_dump-option-file-20210728.patchtext/x-patch; charset=US-ASCII; name=pg_dump-option-file-20210728.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..94fc5aa6c2 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -1006,6 +1006,42 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--options-file=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read options from file (one option per line). Short or long options
+        are supported. If you use "-" as a filename, the filters are read
+        from stdin.
+       </para>
+
+       <para>
+        With the following options file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+-t mytable1
+--include-foreign-data=some_foreign_server
+--exclude-table-data=mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The text after symbol <literal>#</literal> is ignored. This can
+        be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The option <option>--options-file</option> can be used more times,
+        and the nesting is allowed. The options from options files are
+        processed first, other options from command line later. Any option
+        file is processed only one time. In next time the processing is
+        ignored.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--quote-all-identifiers</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 90ac445bcd..29f85ccfbf 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -128,18 +130,34 @@ static SimpleOidList foreign_servers_include_oids = {NULL, NULL};
 static SimpleStringList extension_include_patterns = {NULL, NULL};
 static SimpleOidList extension_include_oids = {NULL, NULL};
 
+static SimpleStringList optsfilenames_processed = {NULL, NULL};
+
 static const CatalogId nilCatalogId = {0, 0};
 
 /* override for standard extra_float_digits setting */
 static bool have_extra_float_digits = false;
 static int	extra_float_digits;
 
+static const char *filename = NULL;
+static const char *format = "p";
+static bool g_verbose = false;
+static const char *dumpencoding = NULL;
+static const char *dumpsnapshot = NULL;
+static char *use_role = NULL;
+static int numWorkers = 1;
+static int compressLevel = -1;
+
 /*
  * The default number of rows per INSERT when
  * --inserts is specified without --rows-per-insert
  */
 #define DUMP_DEFAULT_ROWS_PER_INSERT 1
 
+/*
+ * Option's code of "options-file" option
+ */
+#define OPTIONS_FILE_OPTNUM 12
+
 /*
  * Macro for producing quoted, schema-qualified name of a dumpable object.
  */
@@ -308,14 +326,212 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static bool read_options_from_file(const char *filename,
+								   DumpOptions *dopt,
+								   const char *optstring,
+								   const struct option *longopts,
+								   const char *progname);
+
+/*
+ * It assigns the values of options to related DumpOption fields or to
+ * some global values. Options file loading is not processed here.
+ */
+static bool
+process_option(int opt,
+			   char *optargstr,
+			   DumpOptions *dopt,
+			   const char *progname)
+{
+	switch (opt)
+	{
+		case 'a':			/* Dump data only */
+			dopt->dataOnly = true;
+			break;
+
+		case 'b':			/* Dump blobs */
+			dopt->outputBlobs = true;
+			break;
+
+		case 'B':			/* Don't dump blobs */
+			dopt->dontOutputBlobs = true;
+			break;
+
+		case 'c':			/* clean (i.e., drop) schema prior to create */
+			dopt->outputClean = 1;
+			break;
+
+		case 'C':			/* Create DB */
+			dopt->outputCreateDB = 1;
+			break;
+
+		case 'd':			/* database name */
+			dopt->cparams.dbname = pg_strdup(optargstr);
+			break;
+
+		case 'e':			/* include extension(s) */
+			simple_string_list_append(&extension_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'E':			/* Dump encoding */
+			dumpencoding = pg_strdup(optargstr);
+			break;
+
+		case 'f':
+			filename = pg_strdup(optargstr);
+			break;
+
+		case 'F':
+			format = pg_strdup(optargstr);
+			break;
+
+		case 'h':			/* server host */
+			dopt->cparams.pghost = pg_strdup(optargstr);
+			break;
+
+		case 'j':			/* number of dump jobs */
+			if (!option_parse_int(optarg, "-j/--jobs", 1,
+								  PG_MAX_JOBS,
+								  &numWorkers))
+				exit_nicely(1);
+			break;
+
+		case 'n':			/* include schema(s) */
+			simple_string_list_append(&schema_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'N':			/* exclude schema(s) */
+			simple_string_list_append(&schema_exclude_patterns, optargstr);
+			break;
+
+		case 'O':			/* Don't reconnect to match owner */
+			dopt->outputNoOwner = 1;
+			break;
+
+		case 'p':			/* server port */
+			dopt->cparams.pgport = pg_strdup(optargstr);
+			break;
+
+		case 'R':
+			/* no-op, still accepted for backwards compatibility */
+			break;
+
+		case 's':			/* dump schema only */
+			dopt->schemaOnly = true;
+			break;
+
+		case 'S':			/* Username for superuser in plain text output */
+			dopt->outputSuperuser = pg_strdup(optargstr);
+			break;
+
+		case 't':			/* include table(s) */
+			simple_string_list_append(&table_include_patterns, optargstr);
+			dopt->include_everything = false;
+			break;
+
+		case 'T':			/* exclude table(s) */
+			simple_string_list_append(&table_exclude_patterns, optargstr);
+			break;
+
+		case 'U':
+			dopt->cparams.username = pg_strdup(optargstr);
+			break;
+
+		case 'v':			/* verbose */
+			g_verbose = true;
+			pg_logging_increase_verbosity();
+			break;
+
+		case 'w':
+			dopt->cparams.promptPassword = TRI_NO;
+			break;
+
+		case 'W':
+			dopt->cparams.promptPassword = TRI_YES;
+			break;
+
+		case 'x':			/* skip ACL dump */
+			dopt->aclsSkip = true;
+			break;
+
+		case 'Z':			/* Compression Level */
+			if (!option_parse_int(optarg, "-Z/--compress", 0, 9,
+								  &compressLevel))
+				exit_nicely(1);
+			break;
+
+		case 0:
+			/* This covers the long options. */
+			break;
+
+		case 2:				/* lock-wait-timeout */
+			dopt->lockWaitTimeout = pg_strdup(optargstr);
+			break;
+
+		case 3:				/* SET ROLE */
+			use_role = pg_strdup(optargstr);
+			break;
+
+		case 4:				/* exclude table(s) data */
+			simple_string_list_append(&tabledata_exclude_patterns, optargstr);
+			break;
+
+		case 5:				/* section */
+			set_dump_section(optargstr, &dopt->dumpSections);
+			break;
+
+		case 6:				/* snapshot */
+			dumpsnapshot = pg_strdup(optargstr);
+			break;
+
+		case 7:				/* no-sync */
+			dosync = false;
+			break;
+
+		case 8:
+			have_extra_float_digits = true;
+			if (!option_parse_int(optarg, "--extra-float-digits", -15, 3,
+								  &extra_float_digits))
+				exit_nicely(1);
+			break;
+
+		case 9:				/* inserts */
+
+			/*
+			 * dump_inserts also stores --rows-per-insert, careful not to
+			 * overwrite that.
+			 */
+			if (dopt->dump_inserts == 0)
+				dopt->dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
+			break;
+
+		case 10:			/* rows per insert */
+			if (!option_parse_int(optarg, "--rows-per-insert", 1, INT_MAX,
+								  &dopt->dump_inserts))
+				exit_nicely(1);
+			break;
+
+		case 11:			/* include foreign data */
+			simple_string_list_append(&foreign_servers_include_patterns,
+									  optargstr);
+			break;
+
+		case OPTIONS_FILE_OPTNUM:	/* reading options file */
+			break;					/* should not be processed here ever */
+
+		default:
+			fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			return false;
+	}
 
+	return true;
+}
 
 int
 main(int argc, char **argv)
 {
 	int			c;
-	const char *filename = NULL;
-	const char *format = "p";
 	TableInfo  *tblinfo;
 	int			numTables;
 	DumpableObject **dobjs;
@@ -325,16 +541,9 @@ main(int argc, char **argv)
 	int			optindex;
 	RestoreOptions *ropt;
 	Archive    *fout;			/* the script file */
-	bool		g_verbose = false;
-	const char *dumpencoding = NULL;
-	const char *dumpsnapshot = NULL;
-	char	   *use_role = NULL;
-	int			numWorkers = 1;
-	int			compressLevel = -1;
 	int			plainText = 0;
 	ArchiveFormat archiveFormat = archUnknown;
 	ArchiveMode archiveMode;
-
 	static DumpOptions dopt;
 
 	static struct option long_options[] = {
@@ -401,12 +610,15 @@ main(int argc, char **argv)
 		{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
 		{"no-sync", no_argument, NULL, 7},
 		{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
+		{"options-file", required_argument, NULL, OPTIONS_FILE_OPTNUM},
 		{"rows-per-insert", required_argument, NULL, 10},
 		{"include-foreign-data", required_argument, NULL, 11},
-
+		{"include-foreign-data-file", required_argument, NULL, 17},
 		{NULL, 0, NULL, 0}
 	};
 
+	const char *short_options = "abBcCd:e:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:";
+
 	pg_logging_init(argv[0]);
 	pg_logging_set_level(PG_LOG_WARNING);
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_dump"));
@@ -435,190 +647,33 @@ main(int argc, char **argv)
 
 	InitDumpOptions(&dopt);
 
-	while ((c = getopt_long(argc, argv, "abBcCd:e:E:f:F:h:j:n:N:Op:RsS:t:T:U:vwWxZ:",
+	/*
+	 * We would to process options-file opts before other options.
+	 * This implements higher priority of options from command line.
+	 * Later processed options wins because overwrite result of
+	 * early processed options.
+	 */
+	while ((c = getopt_long(argc, argv, short_options,
 							long_options, &optindex)) != -1)
 	{
-		switch (c)
+		if (c == OPTIONS_FILE_OPTNUM)
 		{
-			case 'a':			/* Dump data only */
-				dopt.dataOnly = true;
-				break;
-
-			case 'b':			/* Dump blobs */
-				dopt.outputBlobs = true;
-				break;
-
-			case 'B':			/* Don't dump blobs */
-				dopt.dontOutputBlobs = true;
-				break;
-
-			case 'c':			/* clean (i.e., drop) schema prior to create */
-				dopt.outputClean = 1;
-				break;
-
-			case 'C':			/* Create DB */
-				dopt.outputCreateDB = 1;
-				break;
-
-			case 'd':			/* database name */
-				dopt.cparams.dbname = pg_strdup(optarg);
-				break;
-
-			case 'e':			/* include extension(s) */
-				simple_string_list_append(&extension_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'E':			/* Dump encoding */
-				dumpencoding = pg_strdup(optarg);
-				break;
-
-			case 'f':
-				filename = pg_strdup(optarg);
-				break;
-
-			case 'F':
-				format = pg_strdup(optarg);
-				break;
-
-			case 'h':			/* server host */
-				dopt.cparams.pghost = pg_strdup(optarg);
-				break;
-
-			case 'j':			/* number of dump jobs */
-				if (!option_parse_int(optarg, "-j/--jobs", 1,
-									  PG_MAX_JOBS,
-									  &numWorkers))
-					exit_nicely(1);
-				break;
-
-			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
-				break;
-
-			case 'O':			/* Don't reconnect to match owner */
-				dopt.outputNoOwner = 1;
-				break;
-
-			case 'p':			/* server port */
-				dopt.cparams.pgport = pg_strdup(optarg);
-				break;
-
-			case 'R':
-				/* no-op, still accepted for backwards compatibility */
-				break;
-
-			case 's':			/* dump schema only */
-				dopt.schemaOnly = true;
-				break;
-
-			case 'S':			/* Username for superuser in plain text output */
-				dopt.outputSuperuser = pg_strdup(optarg);
-				break;
-
-			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
-				dopt.include_everything = false;
-				break;
-
-			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
-				break;
-
-			case 'U':
-				dopt.cparams.username = pg_strdup(optarg);
-				break;
-
-			case 'v':			/* verbose */
-				g_verbose = true;
-				pg_logging_increase_verbosity();
-				break;
-
-			case 'w':
-				dopt.cparams.promptPassword = TRI_NO;
-				break;
-
-			case 'W':
-				dopt.cparams.promptPassword = TRI_YES;
-				break;
-
-			case 'x':			/* skip ACL dump */
-				dopt.aclsSkip = true;
-				break;
-
-			case 'Z':			/* Compression Level */
-				if (!option_parse_int(optarg, "-Z/--compress", 0, 9,
-									  &compressLevel))
-					exit_nicely(1);
-				break;
-
-			case 0:
-				/* This covers the long options. */
-				break;
-
-			case 2:				/* lock-wait-timeout */
-				dopt.lockWaitTimeout = pg_strdup(optarg);
-				break;
-
-			case 3:				/* SET ROLE */
-				use_role = pg_strdup(optarg);
-				break;
-
-			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
-				break;
-
-			case 5:				/* section */
-				set_dump_section(optarg, &dopt.dumpSections);
-				break;
-
-			case 6:				/* snapshot */
-				dumpsnapshot = pg_strdup(optarg);
-				break;
-
-			case 7:				/* no-sync */
-				dosync = false;
-				break;
-
-			case 8:
-				have_extra_float_digits = true;
-				if (!option_parse_int(optarg, "--extra-float-digits", -15, 3,
-									  &extra_float_digits))
-					exit_nicely(1);
-				break;
-
-			case 9:				/* inserts */
-
-				/*
-				 * dump_inserts also stores --rows-per-insert, careful not to
-				 * overwrite that.
-				 */
-				if (dopt.dump_inserts == 0)
-					dopt.dump_inserts = DUMP_DEFAULT_ROWS_PER_INSERT;
-				break;
-
-			case 10:			/* rows per insert */
-				if (!option_parse_int(optarg, "--rows-per-insert", 1, INT_MAX,
-									  &dopt.dump_inserts))
-					exit_nicely(1);
-				break;
-
-			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  optarg);
-				break;
-
-			default:
-				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+			if (!read_options_from_file(optarg, &dopt, short_options,
+										long_options, progname))
 				exit_nicely(1);
 		}
 	}
 
+	/* reset getopt_long index */
+	optind = 1;
+
+	while ((c = getopt_long(argc, argv, short_options,
+							long_options, &optindex)) != -1)
+	{
+		if (!process_option(c, optarg, &dopt, progname))
+			exit_nicely(1);
+	}
+
 	/*
 	 * Non-option argument specifies database name as long as it wasn't
 	 * already specified with -d / --dbname
@@ -1053,6 +1108,7 @@ help(const char *progname)
 	printf(_("  --no-toast-compression       do not dump TOAST compression methods\n"));
 	printf(_("  --no-unlogged-table-data     do not dump unlogged table data\n"));
 	printf(_("  --on-conflict-do-nothing     add ON CONFLICT DO NOTHING to INSERT commands\n"));
+	printf(_("  --options-file=FILENAME      read options from options file\n"));
 	printf(_("  --quote-all-identifiers      quote all identifiers, even if not key words\n"));
 	printf(_("  --rows-per-insert=NROWS      number of rows per INSERT; implies --inserts\n"));
 	printf(_("  --section=SECTION            dump named section (pre-data, data, or post-data)\n"));
@@ -18940,3 +18996,450 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and close input file
+ */
+static void
+invalid_optfile_format(FILE *fp,
+							char *message,
+							char *optname,
+							int optnamelen,
+							char *line,
+							int lineno)
+{
+	Assert(message);
+
+	if (optnamelen > 0)
+	{
+		Assert(optname);
+		pg_log_error(message, optnamelen, optname, lineno);
+	}
+	else
+		pg_log_error(message, lineno);
+
+	if (line)
+		fprintf(stderr, "LINE %d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+}
+
+/*
+ * Reads an option argument from file. Supports double-quoted
+ * strings. In this case multi-line strings are supported.
+ */
+static bool
+read_optarg(FILE *fp,
+			const char *filename,
+			char *str,
+			StringInfo line,
+			StringInfo optargument,
+			char *optname,
+			int optnamelen,
+			bool islongopt,
+			int *lineno)
+{
+	if (*str == '\0' || *str == '#')
+	{
+		if (islongopt)
+			invalid_optfile_format(fp,
+								   "option '--%.*s' requires an argument at line %d",
+									optname, optnamelen,
+									line->data,
+									*lineno);
+		else
+			invalid_optfile_format(fp,
+								   "option '-%.*s' requires an argument at line %d",
+								   optname, optnamelen,
+								   line->data,
+								   *lineno);
+		return false;
+	}
+
+	resetStringInfo(optargument);
+
+	/* simple case */
+	if (*str != '"')
+	{
+		char	   *start = str;
+
+		/* read first white char */
+		while (*str != '\0' && *str != '#')
+		{
+			if (*str == ' ')
+				break;
+			str++;
+		}
+
+		appendBinaryStringInfo(optargument, start, str - start);
+	}
+	else
+	{
+		appendStringInfoChar(optargument, *str++);
+
+		while (1)
+		{
+			int		c = *str++;
+
+			if (c == '\0')
+			{
+				/* multiline string, read next line */
+				if (!pg_get_line_buf(fp, line))
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				if (ferror(fp))
+				{
+					pg_log_error("could not read from file \"%s\": %m",
+								 filename);
+					return false;
+				}
+
+				appendStringInfoChar(optargument, '\n');
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+				*lineno += 1;
+
+				c = *str++;
+			}
+
+			if (c == '\\')
+			{
+				c = *str++;
+
+				if (c == '\0')
+				{
+					invalid_optfile_format(fp,
+												"unexpected end of line at line %d",
+												NULL, 0,
+												NULL,
+												*lineno);
+					return false;
+				}
+
+				switch (c)
+				{
+					case 'n':
+						appendStringInfoChar(optargument, '\n');
+						break;
+
+					case 't':
+						appendStringInfoChar(optargument, '\t');
+						break;
+
+					case '\\':
+						appendStringInfoChar(optargument, '\\');
+						break;
+
+					default:
+						appendStringInfoChar(optargument, '\\');
+						appendStringInfoChar(optargument, c);
+				}
+			}
+			else
+			{
+				appendStringInfoChar(optargument, c);
+
+				if (c == '"')
+				{
+					if (*str == '"')
+						str++;
+					else
+						break;
+				}
+			}
+		}
+	}
+
+	/* check garbage after optarg, but ignore white-space */
+	while (isspace(*str))
+		str++;
+
+	/* at the end there should be EOL or comment symbol */
+	if (*str != '\0' && *str != '#')
+	{
+		invalid_optfile_format(fp,
+							   "unexpected characters after an option's argument at line %d",
+							   NULL, 0,
+							   line->data,
+							   *lineno);
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * Returns true, when long option is defined. Assign code,
+ * and check if option has required argument.
+ */
+static bool
+is_recognized_longopt(const struct option *longopts,
+					  char *optname,
+					  int optnamelen,
+					  int *opt,
+					  bool *has_arg)
+{
+	int		i;
+
+	for (i = 0; longopts[i].name != NULL; i++)
+	{
+		if (strlen(longopts[i].name) == optnamelen &&
+			strncmp(optname, longopts[i].name, optnamelen) == 0)
+		{
+			*has_arg = longopts[i].has_arg == required_argument;
+
+			if (longopts[i].flag == NULL)
+			{
+				*opt = longopts[i].val;
+
+				return true;
+			}
+			else
+			{
+				*longopts[i].flag = longopts[i].val;
+				*opt = 0;
+
+				return true;
+			}
+		}
+	}
+
+	return false;
+}
+
+/*
+ * Returns true, when short option char is defined.
+ */
+static bool
+is_valid_shortopt(const char *optstring,
+				  char optname,
+				  bool *has_arg)
+{
+	while (*optstring != '\0')
+	{
+		if (*optstring != ':' && *optstring == optname)
+		{
+			*has_arg = optstring[1] == ':';
+			return true;
+		}
+
+		optstring++;
+	}
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file. Returns
+ * true, when processing was ok.
+ */
+static bool
+read_options_from_file(const char *filename,
+					   DumpOptions *dopt,
+					   const char *optstring,
+					   const struct option *longopts,
+					   const char *progname)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	StringInfoData optargument;
+
+	/* Ignore already processed files */
+	if (simple_string_list_member(&optsfilenames_processed,
+								  filename))
+	{
+		pg_log_warning("the options file \"%s\" was processed already, skip this",
+					   filename);
+		return true;
+	}
+
+	simple_string_list_append(&optsfilenames_processed, filename);
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+		{
+			pg_log_error("could not open the input file \"%s\": %m",
+				  filename);
+			return false;
+		}
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+	initStringInfo(&optargument);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		char	   *optname;
+		char	   *str = line.data;
+		int			opt;
+		int			optnamelen;
+		bool		has_arg;
+
+		(void) pg_strip_crlf(str);
+
+		lineno += 1;
+
+		/* skip initial spaces */
+		while (isspace(*str))
+			str++;
+
+		/* Ignore empty lines or lines with hash symbol (comment) */
+		if (*str == '\0' || *str == '#')
+			continue;
+
+		if (*str++ != '-')
+		{
+			invalid_optfile_format(fp,
+								   "non option arguments are not allowed in options file at line %d",
+								   NULL, 0,
+								   line.data,
+								   lineno);
+			return false;
+		}
+
+		if (*str == '-')
+		{
+			/* process long option */
+			str++;
+			optname = str++;
+
+			while (!isspace(*str) && *str != '=' && *str != '\0')
+				str++;
+
+			optnamelen = str - optname;
+
+			if (is_recognized_longopt(longopts, optname, optnamelen,
+									  &opt, &has_arg))
+			{
+				/* skip optional spaces */
+				while (isspace(*str))
+					str++;
+
+				if (has_arg)
+				{
+					/* skip optional = */
+					if (*str == '=')
+						str++;
+
+					/* skip optional spaces */
+					while (isspace(*str))
+						str++;
+
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, true, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '--%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "unrecognized option '--%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+		else
+		{
+			/* process short option */
+			optname = str++;
+			opt = *optname;
+
+			optnamelen = 1;
+
+			/* skip optional spaces */
+			while (isspace(*str))
+				str++;
+
+			if (is_valid_shortopt(optstring, opt, &has_arg))
+			{
+				if (has_arg)
+				{
+					if (!read_optarg(fp, filename, str, &line, &optargument,
+									 optname, optnamelen, false, &lineno))
+						return false;
+				}
+				else
+				{
+					if (*str != '\0' && *str != '#')
+					{
+						invalid_optfile_format(fp,
+											   "option '-%.*s' doesn't allow an argument at line %d",
+											   optname, optnamelen,
+											   line.data,
+											   lineno);
+						return false;
+					}
+				}
+			}
+			else
+			{
+				invalid_optfile_format(fp,
+									   "invalid option '-%.*s' at line %d",
+									   optname, optnamelen,
+									   line.data,
+									   lineno);
+				return false;
+			}
+		}
+
+		/* nested options file reading */
+		if (opt == OPTIONS_FILE_OPTNUM)
+		{
+			if (!read_options_from_file(optargument.data, dopt, optstring,
+										longopts, progname))
+			{
+				fclose(fp);
+				return false;
+			}
+		}
+		else if (opt != 0 &&
+				 !process_option(opt, optargument.data, dopt, progname))
+		{
+			fclose(fp);
+			return false;
+		}
+	}
+
+	pfree(line.data);
+	pfree(optargument.data);
+
+	if (ferror(fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", filename);
+		return false;
+	}
+
+	if (fp != stdin)
+		fclose(fp);
+
+	return true;
+}
#92Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#79)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

út 13. 7. 2021 v 1:16 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

[1] your proposal of "[+-] OBJTYPE OBJIDENT" plus empty lines allowed
plus lines starting with # are comments, seems plenty. Any line not
following that format would cause an error to be thrown.

I'd like to see some kind of keyword on each line, so that we could extend
the command set by adding new keywords. As this stands, I fear we'd end
up using random punctuation characters in place of [+-], which seems
pretty horrid from a readability standpoint.

I think that this file format should be designed with an eye to allowing
every, or at least most, pg_dump options to be written in the file rather
than on the command line. I don't say we have to *implement* that right
now; but if the format spec is incapable of being extended to meet
requests like that one, I think we'll regret it. This line of thought
suggests that the initial commands ought to match the existing
include/exclude switches, at least approximately.

Hence I suggest

include table PATTERN
exclude table PATTERN

which ends up being the above but with words not [+-].

Here is an updated implementation of filter's file, that implements syntax
proposed by you.

Regards

Pavel

Show quoted text

regards, tom lane

Attachments:

pg_dump-filter-20210728.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20210728.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..d0459b385e 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,56 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.
+        The lines of this file must have the following format:
+<synopsis>
+(include|exclude)[table|schema|foreign_data|data] <replaceable class="parameter">objectname</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included
+        or excluded, and the second keyword specifies the type of object
+        to be filtered:
+        <literal>table</literal> (table),
+        <literal>schema</literal> (schema),
+        <literal>foreign_data</literal> (foreign server),
+        <literal>data</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable1
+include foreign_data some_foreign_server
+exclude table mytable2
+</programlisting>
+       </para>
+
+       <para>
+        The lines starting with symbol <literal>#</literal> are ignored.
+        Previous white chars (spaces, tabs) are not allowed. These
+        lines can be used for comments, notes. Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 90ac445bcd..ba4c425ee6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -308,7 +310,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
-
+static void read_patterns_from_file(char *filename, DumpOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -380,6 +382,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +616,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_patterns_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1045,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18940,3 +18949,281 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FILE *fp, char *filename, char *message, char *line, int lineno)
+{
+	pg_log_error("invalid format of filter file \"%s\": %s",
+				 filename,
+				 message);
+
+	fprintf(stderr, "%d: %s\n", lineno, line);
+
+	if (fp != stdin)
+		fclose(fp);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isblank(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+static bool
+is_keyword(const char *keyword, int size, const char *str)
+{
+	if (strlen(str) != size)
+		return false;
+
+	return pg_strncasecmp(keyword, str, size) == 0;
+}
+
+static bool
+isblank_line(const char *line)
+{
+	while (*line)
+	{
+		if (!isblank(*line++))
+			return false;
+	}
+
+	return true;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_patterns_from_file(char *filename, DumpOptions *dopt)
+{
+	FILE	   *fp;
+	int			lineno = 0;
+	StringInfoData line;
+	PQExpBuffer quoted_name = NULL;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fp = fopen(filename, "r");
+		if (!fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fp = stdin;
+
+	initStringInfo(&line);
+
+	while (pg_get_line_buf(fp, &line))
+	{
+		bool		is_include;
+		char		objecttype;
+		char	   *objectname;
+		char	   *str = line.data;
+		char	   *str_mark;
+		const char	   *keyword;
+		int			size;
+
+		lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* ignore blank lines */
+		if (isblank_line(str))
+			continue;
+
+		/* when first char is hash, ignore whole line */
+		if (*str == '#')
+			continue;
+
+		keyword = get_keyword((const char **) &str, &size);
+
+		/* Now we expect sequence of two keywords */
+		if (keyword && is_keyword(keyword, size, "include"))
+			is_include = true;
+		else if (keyword && is_keyword(keyword, size, "exclude"))
+			is_include = false;
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "expected keyword \"include\" or \"exclude\"",
+									   line.data,
+									   lineno);
+
+		/*
+		 * Save current position in parsed line. Can be used later
+		 * in error message.
+		 */
+		str_mark = str;
+
+		keyword = get_keyword((const char **) &str, &size);
+
+		if (keyword && is_keyword(keyword, size, "table"))
+			objecttype = 't';
+		else if (keyword && is_keyword(keyword, size, "schema"))
+			objecttype = 's';
+		else if (keyword && is_keyword(keyword, size, "foreign_data"))
+			objecttype = 'f';
+		else if (keyword && is_keyword(keyword, size, "data"))
+			objecttype = 'd';
+		else
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "expected keyword \"table\", \"schema\", \"foreign_data\" or \"data\"",
+									   str_mark,
+									   lineno);
+
+		objectname = str;
+
+		/* skip initial spaces */
+		while (isspace(*objectname))
+			objectname++;
+
+		if (*objectname == '\0')
+			exit_invalid_filter_format(fp,
+									   filename,
+									   "missing object name",
+									   str,
+									   lineno);
+
+		if (*objectname == '"')
+		{
+			PQExpBuffer		quoted_name;
+			char	   *ptr = objectname + 1;
+
+			quoted_name = createPQExpBuffer();
+
+			appendPQExpBufferChar(quoted_name, '"');
+
+			while (1)
+			{
+				if (*ptr == '\0')
+				{
+					if (!pg_get_line_buf(fp, &line))
+						exit_invalid_filter_format(fp,
+												   filename,
+												   "unexpected end of file",
+												   "",
+												   lineno);
+
+					if (ferror(fp))
+						fatal("could not read from file \"%s\": %m", filename);
+
+					appendPQExpBufferChar(quoted_name, '\n');
+					ptr = line.data;
+					lineno += 1;
+				}
+
+				appendPQExpBufferChar(quoted_name, *ptr);
+				if (*ptr++ == '"')
+				{
+					if (*ptr == '"')
+						appendPQExpBufferChar(quoted_name, *ptr++);
+					else
+						break;
+				}
+			}
+
+			/* check garbage after identifier */
+			if (!isblank_line(ptr))
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "unexpected chars after object name",
+										   ptr,
+										   lineno);
+
+			objectname = quoted_name->data;
+		}
+
+		if (objecttype == 't')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 's')
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objectname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'd')
+		{
+			if (is_include)
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "include filter is not supported for this type of object",
+										   str,
+										   lineno);
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objectname);
+		}
+		else if (objecttype == 'f')
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objectname);
+			else
+				exit_invalid_filter_format(fp,
+										   filename,
+										   "exclude filter is not supported for this type of object",
+										   str,
+										   lineno);
+		}
+
+		if (quoted_name)
+		{
+			destroyPQExpBuffer(quoted_name);
+			quoted_name = NULL;
+		}
+	}
+
+	pfree(line.data);
+
+	if (ferror(fp))
+		fatal("could not read from file \"%s\": %m", filename);
+
+	if (fp != stdin)
+		fclose(fp);
+}
#93Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#92)
Re: proposal: possibility to read dumped table's name from file

On 28 Jul 2021, at 09:28, Pavel Stehule <pavel.stehule@gmail.com> wrote:
út 13. 7. 2021 v 1:16 odesílatel Tom Lane <tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> napsal:

Hence I suggest

include table PATTERN
exclude table PATTERN

which ends up being the above but with words not [+-].

One issue with this syntax is that the include keyword can be quite misleading
as it's semantic interpretion of "include table t" can be different from
"--table=t". The former is less clear about the fact that it means "exclude
all other tables than " then the latter. It can be solved with documentation,
but I think that needs be to be made clearer.

Here is an updated implementation of filter's file, that implements syntax proposed by you.

While it's not the format I would prefer, it does allow for most (all?) use
cases expressed in this thread with ample armtwisting applied so let's go ahead
from this point and see if we can agree on it (or a version of it).

A few notes on the patch after a first pass over it:

+(include|exclude)[table|schema|foreign_data|data] <replaceable class="parameter">objectname</replaceable>
Lacks whitespace between keyword and object type. Also, since these are
mandatory parameters, shouldn't they be within '{..}' ?

+	/* skip initial white spaces */
+	while (isblank(*ptr))
+		ptr += 1;
We don't trust isblank() as of 3fd5faed5 due to portability concerns, this
should probably use a version of the pg_isblank() we already have (and possibly
move that to src/common/string.c as there now are more consumers).
+static bool
+isblank_line(const char *line)
This could be replaced with a single call to strspn() as we already do for
parsing the TOC file.
+	/* when first char is hash, ignore whole line */
+	if (*str == '#')
+		continue;
I think we should strip leading whitespace before this to allow commentlines to
start with whitespace, it's easy enough and will make life easier for users.
+       pg_log_error("invalid format of filter file \"%s\": %s",
+                                filename,
+                                message);
+
+       fprintf(stderr, "%d: %s\n", lineno, line);
Can't we just include the lineno in the error logging and skip dumping the
offending line?  Fast-forwarding the pointer to print the offending part is
less useful than a context marker, and in some cases suboptimal.  With this
coding, if a pattern is omitted for example the below error message is given:

pg_dump: error: invalid format of filter file "filter.txt": missing object name
1:

The errormessage and the linenumber in the file should be enough for the user
to figure out what to fix.

+       if (keyword && is_keyword(keyword, size, "table"))
+               objecttype = 't';
Should this use an enum, or at least a struct translation the literal keyword
to the internal representation?  Magic constants without explicit connection to
their token counterparts can easily be cause of bugs.

If I create a table called "a\nb" and try to dump it I get an error in parsing
the file. Isn't this supposed to work?
$ cat filter.txt
include table "a
b"
$ ./bin/pg_dump --filter=filter.txt
pg_dump: error: invalid format of filter file "filter.txt": unexpected chars after object name
2:

Did you consider implementing this in Bison to abstract some of the messier
parsing logic?

--
Daniel Gustafsson https://vmware.com/

#94Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#92)
Re: proposal: possibility to read dumped table's name from file

On Wed, Jul 28, 2021 at 09:28:17AM +0200, Pavel Stehule wrote:

Here is an updated implementation of filter's file, that implements syntax
proposed by you.

Thanks.

If there's any traction for this approach. I have some comments for the next
revision,

+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,56 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.

Say 'Specify "-" to read from stdin'

+ The lines starting with symbol <literal>#</literal> are ignored.

Remove "The" and "symbol"

+ Previous white chars (spaces, tabs) are not allowed. These

Preceding whitespace characters...

But actually, they are allowed? But if it needs to be explained, maybe they
shouldn't be - I don't see the utility of it.

+static bool
+isblank_line(const char *line)
+{
+	while (*line)
+	{
+		if (!isblank(*line++))
+			return false;
+	}
+
+	return true;
+}

I don't think this requires nor justifies having a separate function.
Either don't support blank lines, or use get_keyword() with size==0 for that ?

+		/* Now we expect sequence of two keywords */
+		if (keyword && is_keyword(keyword, size, "include"))
+			is_include = true;
+		else if (keyword && is_keyword(keyword, size, "exclude"))
+			is_include = false;
+		else

I think this should first check "if keyword == NULL".
That could give a more specific error message like "no keyword found",

+			exit_invalid_filter_format(fp,
+									   filename,
+									   "expected keyword \"include\" or \"exclude\"",
+									   line.data,
+									   lineno);

..and then this one can say "invalid keyword".

--
Justin

#95Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#93)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

po 13. 9. 2021 v 15:01 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 28 Jul 2021, at 09:28, Pavel Stehule <pavel.stehule@gmail.com> wrote:
út 13. 7. 2021 v 1:16 odesílatel Tom Lane <tgl@sss.pgh.pa.us <mailto:

tgl@sss.pgh.pa.us>> napsal:

Hence I suggest

include table PATTERN
exclude table PATTERN

which ends up being the above but with words not [+-].

One issue with this syntax is that the include keyword can be quite
misleading
as it's semantic interpretion of "include table t" can be different from
"--table=t". The former is less clear about the fact that it means
"exclude
all other tables than " then the latter. It can be solved with
documentation,
but I think that needs be to be made clearer.

I invite any documentation enhancing and fixing

Here is an updated implementation of filter's file, that implements

syntax proposed by you.

While it's not the format I would prefer, it does allow for most (all?) use
cases expressed in this thread with ample armtwisting applied so let's go
ahead
from this point and see if we can agree on it (or a version of it).

A few notes on the patch after a first pass over it:

+(include|exclude)[table|schema|foreign_data|data] <replaceable
class="parameter">objectname</replaceable>
Lacks whitespace between keyword and object type. Also, since these are
mandatory parameters, shouldn't they be within '{..}' ?

yes, fixed

+ /* skip initial white spaces */
+ while (isblank(*ptr))
+ ptr += 1;
We don't trust isblank() as of 3fd5faed5 due to portability concerns, this
should probably use a version of the pg_isblank() we already have (and
possibly
move that to src/common/string.c as there now are more consumers).

I rewrote this part, and I don't use function isblank ever

+static bool
+isblank_line(const char *line)
This could be replaced with a single call to strspn() as we already do for
parsing the TOC file.
+       /* when first char is hash, ignore whole line */
+       if (*str == '#')
+               continue;
I think we should strip leading whitespace before this to allow
commentlines to
start with whitespace, it's easy enough and will make life easier for
users.

now, the comments can be used as first non blank char or after filter

+       pg_log_error("invalid format of filter file \"%s\": %s",
+                                filename,
+                                message);
+
+       fprintf(stderr, "%d: %s\n", lineno, line);
Can't we just include the lineno in the error logging and skip dumping the
offending line?  Fast-forwarding the pointer to print the offending part is
less useful than a context marker, and in some cases suboptimal.  With this
coding, if a pattern is omitted for example the below error message is
given:

pg_dump: error: invalid format of filter file "filter.txt": missing

object name
1:

The errormessage and the linenumber in the file should be enough for the
user
to figure out what to fix.

I did it like you proposed, but still, I think the content can be useful.
More times you read dynamically generated files, or you read data from
stdin, and in complex environments it can be hard regenerate new content
for debugging.

+       if (keyword && is_keyword(keyword, size, "table"))
+               objecttype = 't';
Should this use an enum, or at least a struct translation the literal
keyword
to the internal representation?  Magic constants without explicit
connection to
their token counterparts can easily be cause of bugs.

fixed

If I create a table called "a\nb" and try to dump it I get an error in
parsing
the file. Isn't this supposed to work?
$ cat filter.txt
include table "a
b"
$ ./bin/pg_dump --filter=filter.txt
pg_dump: error: invalid format of filter file "filter.txt": unexpected
chars after object name
2:

probably there was some issue, because it should work. I tested a new
version and this is tested in new regress tests. Please, check

Did you consider implementing this in Bison to abstract some of the messier
parsing logic?

Initially not, but now, when I am thinking about it, I don't think so Bison
helps. The syntax of the filter file is nicely linear. Now, the code of the
parser is a little bit larger than minimalistic, but it is due to nicer
error's messages. The raw implementation in Bison raised just "syntax
error" and positions. I did code refactoring, and now the scanning, parsing
and processing are divided into separated routines. Parsing related code
has 90 lines. In this case, I don't think using a parser grammar file can
carry any benefit. grammar is more readable, sure, but we need to include
bison, we need to handle errors, and if we want to raise more helpful
errors than just "syntax error", then the code will be longer.

please, check attached patch

Regards

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20210915.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20210915.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..1b74c0eadd 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,55 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file. Specify "-" to read from
+        stdin. Lines of this file must have the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included
+        or excluded, and the second keyword specifies the type of object
+        to be filtered:
+        <literal>table</literal> (table),
+        <literal>schema</literal> (schema),
+        <literal>foreign_data</literal> (foreign server),
+        <literal>data</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable1
+include foreign_data some_foreign_server
+exclude table mytable2
+</programlisting>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are ignored. The comment
+        (started by <literal>#</literal>) can be placed after filter too.
+        Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..844492c64f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -308,7 +310,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
-
+static void read_filters_from_file(char *filename, DumpOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -380,6 +382,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +616,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_filters_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1045,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +18988,343 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+typedef struct
+{
+	FILE	   *fp;
+	char	   *filename;
+	int			lineno;
+} FilterStateData;
+
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		fclose(fstate->fp);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isspace(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+static bool
+filter_is_keyword(const char *keyword, int size, const char *str)
+{
+	if (strlen(str) != size)
+		return false;
+
+	return pg_strncasecmp(keyword, str, size) == 0;
+}
+
+/*
+ * Sets objname to string with object identifier. The line variable holds string
+ * of last line with object identifier (object name). Returns pointer to first char
+ * after last char of object name.
+ */
+static char *
+filter_get_object_name(FilterStateData *fstate,
+					   StringInfo line,
+					   char *str,
+					   char **objname)
+{
+	/* skip white spaces */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name");
+
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (!pg_get_line_buf(fstate->fp, line))
+				{
+					if (ferror(fstate->fp))
+						fatal("could not read from file \"%s\": %m", fstate->filename);
+
+					exit_invalid_filter_format(fstate,"unexpected end of file");
+				}
+
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				str = line->data;
+				fstate->lineno += 1;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+					str++;
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = quoted_name->data;
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* simple variant, read to end or to first space */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = strndup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * Returns true, when one filter item was successfully read and parsed.
+ * When object name contains \n chars, then more than one line from input
+ * file can be processed. Returns false when EOF. Run exit on error.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * skip empty lines or lines when first noblank char is hash (comment)
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"include\" or \"exclude\")");
+
+			/* Now we expect sequence of two keywords */
+			if (filter_is_keyword(keyword, size, "include"))
+				*is_include = true;
+			else if (filter_is_keyword(keyword, size, "exclude"))
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (filter_is_keyword(keyword, size, "table"))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (filter_is_keyword(keyword, size, "schema"))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (filter_is_keyword(keyword, size, "foreign_data"))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (filter_is_keyword(keyword, size, "data"))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_object_name(fstate, &line, str, objname);
+
+			/*
+			 * check possible content after object identifier.
+			 * Allow comment started by hash.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+				   "unexpected chars after object name");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_fatal("could not read from file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+			fclose(fstate->fp);
+
+		exit_nicely(-1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_filters_from_file(char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		free(objname);
+	}
+
+	if (fstate.fp != stdin)
+		fclose(fstate.fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..505614f145
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,151 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 24;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"broken format check");
#96Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#94)
Re: proposal: possibility to read dumped table's name from file

po 13. 9. 2021 v 15:11 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Wed, Jul 28, 2021 at 09:28:17AM +0200, Pavel Stehule wrote:

Here is an updated implementation of filter's file, that implements

syntax

proposed by you.

Thanks.

If there's any traction for this approach. I have some comments for the
next
revision,

+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,56 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable

class="parameter">filename</replaceable></option></term>

+      <listitem>
+       <para>
+        Read objects filters from the specified file.
+        If you use "-" as a filename, the filters are read from stdin.

Say 'Specify "-" to read from stdin'

+ The lines starting with symbol <literal>#</literal> are ignored.

Remove "The" and "symbol"

+ Previous white chars (spaces, tabs) are not allowed. These

Preceding whitespace characters...

But actually, they are allowed? But if it needs to be explained, maybe
they
shouldn't be - I don't see the utility of it.

+static bool
+isblank_line(const char *line)
+{
+     while (*line)
+     {
+             if (!isblank(*line++))
+                     return false;
+     }
+
+     return true;
+}

I don't think this requires nor justifies having a separate function.
Either don't support blank lines, or use get_keyword() with size==0 for
that ?

+             /* Now we expect sequence of two keywords */
+             if (keyword && is_keyword(keyword, size, "include"))
+                     is_include = true;
+             else if (keyword && is_keyword(keyword, size, "exclude"))
+                     is_include = false;
+             else

I think this should first check "if keyword == NULL".
That could give a more specific error message like "no keyword found",

+                     exit_invalid_filter_format(fp,
+

filename,

+

"expected keyword \"include\" or \"exclude\"",

+

line.data,

+

lineno);

..and then this one can say "invalid keyword".

I fixed (I hope) mentioned issues. Please check last patch

Regards

Pavel

Show quoted text

--
Justin

#97Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#95)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

In yesterday's patch I used strndup, which is not available on win. I am
sending update when I used pnstrdup instead.

Regards

Pavel

Attachments:

pg_dump-filteropt-20210916.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20210916.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..1b74c0eadd 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,55 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file. Specify "-" to read from
+        stdin. Lines of this file must have the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included
+        or excluded, and the second keyword specifies the type of object
+        to be filtered:
+        <literal>table</literal> (table),
+        <literal>schema</literal> (schema),
+        <literal>foreign_data</literal> (foreign server),
+        <literal>data</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable1
+include foreign_data some_foreign_server
+exclude table mytable2
+</programlisting>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are ignored. The comment
+        (started by <literal>#</literal>) can be placed after filter too.
+        Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..81533eb144 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -308,7 +310,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
-
+static void read_filters_from_file(char *filename, DumpOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -380,6 +382,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +616,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_filters_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1045,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +18988,344 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+typedef struct
+{
+	FILE	   *fp;
+	char	   *filename;
+	int			lineno;
+} FilterStateData;
+
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		fclose(fstate->fp);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isspace(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+static bool
+filter_is_keyword(const char *keyword, int size, const char *str)
+{
+	if (strlen(str) != size)
+		return false;
+
+	return pg_strncasecmp(keyword, str, size) == 0;
+}
+
+/*
+ * Sets objname to string with object identifier. The line variable holds string
+ * of last line with object identifier (object name). Returns pointer to first char
+ * after last char of object name.
+ */
+static char *
+filter_get_object_name(FilterStateData *fstate,
+					   StringInfo line,
+					   char *str,
+					   char **objname)
+{
+	/* skip white spaces */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name");
+
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (!pg_get_line_buf(fstate->fp, line))
+				{
+					if (ferror(fstate->fp))
+						fatal("could not read from file \"%s\": %m", fstate->filename);
+
+					exit_invalid_filter_format(fstate,"unexpected end of file");
+				}
+
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				str = line->data;
+				fstate->lineno += 1;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+					str++;
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = quoted_name->data;
+	}
+	else
+	{
+		char	   *startptr = str++;
+		size_t		objnamelen;
+
+		/* simple variant, read to end or to first space */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * Returns true, when one filter item was successfully read and parsed.
+ * When object name contains \n chars, then more than one line from input
+ * file can be processed. Returns false when EOF. Run exit on error.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * skip empty lines or lines when first noblank char is hash (comment)
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"include\" or \"exclude\")");
+
+			/* Now we expect sequence of two keywords */
+			if (filter_is_keyword(keyword, size, "include"))
+				*is_include = true;
+			else if (filter_is_keyword(keyword, size, "exclude"))
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (filter_is_keyword(keyword, size, "table"))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (filter_is_keyword(keyword, size, "schema"))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (filter_is_keyword(keyword, size, "foreign_data"))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (filter_is_keyword(keyword, size, "data"))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_object_name(fstate, &line, str, objname);
+
+			/*
+			 * check possible content after object identifier.
+			 * Allow comment started by hash.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+				   "unexpected chars after object name");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_fatal("could not read from file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+			fclose(fstate->fp);
+
+		exit_nicely(-1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_filters_from_file(char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		free(objname);
+	}
+
+	if (fstate.fp != stdin)
+		fclose(fstate.fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..505614f145
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,151 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 24;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"broken format check");
#98Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#97)
Re: proposal: possibility to read dumped table's name from file

As there have been a lot of differing opinions raised in this thread, I re-read
it and tried to summarize the discussion so far to try and figure out where we
agree and on what (and disagree) before we get deep into the technicalities wrt
the current patch. If anyone feel I've misrepresented them below then I
sincerely do apologize. If I missed a relevant viewpoint I also apologize,
I've tried to objectively represent the thread.

I proposed JSON in [0]/messages/by-id/F6674FF0-5800-4AED-9DC7-13C475707241@yesql.se which is where the format discussion to some extent
started, Justin and Pavel had up until that point discussed the format by
refining the original proposal.

In [1]/messages/by-id/CALAY4q9u30L7oGhbsfY3dPECQ8SrYa8YO=H-xOn5xWUeiEneeg@mail.gmail.com Surafel Temesgen brought up --exclude-database from pg_dumpall and
--no-comments, and argued for them being handled by this patch. This was
objected against on the grounds that pg_dumpall is out of scope, and
all-or-nothing switches not being applicable in a filter option.

Stephen objected to both the proposed, and the suggestion of JSON, in [2]/messages/by-id/20201110200904.GU16415@tamriel.snowman.net and
argued for a more holistic configuration file approach. TOML was suggested.
Dean then +1'd the config file approach in [3]/messages/by-id/CAEZATCVKMG7+b+_5tNwrNZ-aNDBy3=FMRNea2bO9O4qGcEvSTg@mail.gmail.com.

In [4]/messages/by-id/502641.1606334432@sss.pgh.pa.us Tom supported the idea of a more generic config file, and remarked that
the proposed filter for table names only makes sense when the number of exclude
patterns are large enough that we might hit other problems in pg_dump.
Further, in [5]/messages/by-id/619671.1606406538@sss.pgh.pa.us Tom commented that a format with established quoting
conventions would buy us not having to invent our own to cope with complicated
relation names.

The fact that JSON doesn't support comments is brought up in a few emails and
is a very valid point, as the need for comments regardless of format is brought
up as well.

Tomas Vondra in [6]/messages/by-id/cb545d78-2dae-8d27-f062-822a07ca56cf@enterprisedb.com wanted the object filter be a separate file from a config
file, and argued for a simpler format for these lists (while still supporting
multiple object types).

Alvaro agreed with Tomas on [+-] OBJTYPE OBJIDENT in [7]/messages/by-id/202107122259.n6o5uwb5erza@alvherre.pgsql and Tom extended the
proposal to use [include/exclude] keywords in [8]/messages/by-id/3183720.1626131795@sss.pgh.pa.us in order to support more than
just excluding and including. Regardless of stance on format, the use of
keywords instead of [+-] is a rare point of consensus in this thread.

Stephen and myself have also expressed concern in various parts of the thread
that inventing our own format rather than using something with existing broad
library support will end up with third-parties (like pgAdmin et.al) having to
all write their own generators and parsers.

A main concern among most (all?) participants of the thread, regardless of
format supported, is that quoting is hard and must be done right for all object
names postgres support (including any not currently in scope by this patch).

Below is an attempt at summarizing and grouping the proposals so far into the
set of ideas presented:

A) A keyword+object based format to invoke with a switch to essentially
allow for more filters than the commandline can handle and nothing more.
After a set of revisions, the current proposal is:
[include|exclude] [<objtype>] [<objident>]

B) A format similar to (A) which can also be used for pg_dump configuration

C) The format in (A), or a close variant thereof, with the intention of it
being included in/referred to from a future configuration file of currently
unknown format. One reference being a .gitignore type file.

D) An existing format (JSON and TOML have been suggested, with JSON
being dismissed due to lack of comment support) which has quoting
conventions that supports postgres' object names and which can be used to
define a full pg_dump configuration file syntax.

For B), C) and D) there is implicit consensus in the thread that we don't need
to implement the full configuration file as of this patch, merely that it
*must* be possible to do so without having to paint ourselves out of a corner.

At this point it seems to me that B) and C) has the broadest support. Can the
C) option may represent the compromise between "simple" format for object
filtering and a more structured format for configuration? Are there other
options?

Thoughts?

--
Daniel Gustafsson https://vmware.com/

[0]: /messages/by-id/F6674FF0-5800-4AED-9DC7-13C475707241@yesql.se
[1]: /messages/by-id/CALAY4q9u30L7oGhbsfY3dPECQ8SrYa8YO=H-xOn5xWUeiEneeg@mail.gmail.com
[2]: /messages/by-id/20201110200904.GU16415@tamriel.snowman.net
[3]: /messages/by-id/CAEZATCVKMG7+b+_5tNwrNZ-aNDBy3=FMRNea2bO9O4qGcEvSTg@mail.gmail.com
[4]: /messages/by-id/502641.1606334432@sss.pgh.pa.us
[5]: /messages/by-id/619671.1606406538@sss.pgh.pa.us
[6]: /messages/by-id/cb545d78-2dae-8d27-f062-822a07ca56cf@enterprisedb.com
[7]: /messages/by-id/202107122259.n6o5uwb5erza@alvherre.pgsql
[8]: /messages/by-id/3183720.1626131795@sss.pgh.pa.us

#99Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#95)
Re: proposal: possibility to read dumped table's name from file

On 15 Sep 2021, at 19:31, Pavel Stehule <pavel.stehule@gmail.com> wrote:
po 13. 9. 2021 v 15:01 odesílatel Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se>> napsal:

One issue with this syntax is that the include keyword can be quite misleading
as it's semantic interpretion of "include table t" can be different from
"--table=t". The former is less clear about the fact that it means "exclude
all other tables than " then the latter. It can be solved with documentation,
but I think that needs be to be made clearer.

I invite any documentation enhancing and fixing

Sure, that can be collabored on. This gist is though that IMO the keywords in
the filter file aren't as clear on the sideeffects as the command line params,
even though they are equal in functionality.

+       pg_log_error("invalid format of filter file \"%s\": %s",
+                                filename,
+                                message);
+
+       fprintf(stderr, "%d: %s\n", lineno, line);
Can't we just include the lineno in the error logging and skip dumping the
offending line?  Fast-forwarding the pointer to print the offending part is
less useful than a context marker, and in some cases suboptimal.  With this
coding, if a pattern is omitted for example the below error message is given:

pg_dump: error: invalid format of filter file "filter.txt": missing object name
1:

The errormessage and the linenumber in the file should be enough for the user
to figure out what to fix.

I did it like you proposed, but still, I think the content can be useful.

Not when there is no content in the error message, printing an empty string for
a line number which isn't a blank line doesn't seem terribly helpful. If we
know the error context is empty, printing a tailored error hint seems more
useful for the user.

More times you read dynamically generated files, or you read data from stdin, and in complex environments it can be hard regenerate new content for debugging.

That seems odd given that the arguments for this format has been that it's
likely to be handwritten.

If I create a table called "a\nb" and try to dump it I get an error in parsing
the file. Isn't this supposed to work?
$ cat filter.txt
include table "a
b"
$ ./bin/pg_dump --filter=filter.txt
pg_dump: error: invalid format of filter file "filter.txt": unexpected chars after object name
2:

probably there was some issue, because it should work. I tested a new version and this is tested in new regress tests. Please, check

That seems to work, but I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

Did you consider implementing this in Bison to abstract some of the messier
parsing logic?

Initially not, but now, when I am thinking about it, I don't think so Bison helps. The syntax of the filter file is nicely linear. Now, the code of the parser is a little bit larger than minimalistic, but it is due to nicer error's messages. The raw implementation in Bison raised just "syntax error" and positions. I did code refactoring, and now the scanning, parsing and processing are divided into separated routines. Parsing related code has 90 lines. In this case, I don't think using a parser grammar file can carry any benefit. grammar is more readable, sure, but we need to include bison, we need to handle errors, and if we want to raise more helpful errors than just "syntax error", then the code will be longer.

I'm not so concerned by code size, but rather parsing of quotations etc and
being able to reason about it's correctness. IMHO that's easier done by
reading a defined grammar than parsing a handwritten parser.

Will do a closer review on the patch shortly.

--
Daniel Gustafsson https://vmware.com/

#100Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#98)
Re: proposal: possibility to read dumped table's name from file

Hi

A main concern among most (all?) participants of the thread, regardless of
format supported, is that quoting is hard and must be done right for all
object
names postgres support (including any not currently in scope by this
patch).

Just a small note - when quoting is calculated to design, then the
implementation is easy. I am sure, so my last code covers all
possibilities, and it is about 100 lines of code.

Below is an attempt at summarizing and grouping the proposals so far into
the
set of ideas presented:

A) A keyword+object based format to invoke with a switch to essentially
allow for more filters than the commandline can handle and nothing
more.
After a set of revisions, the current proposal is:
[include|exclude] [<objtype>] [<objident>]

B) A format similar to (A) which can also be used for pg_dump
configuration

C) The format in (A), or a close variant thereof, with the intention
of it
being included in/referred to from a future configuration file of
currently
unknown format. One reference being a .gitignore type file.

D) An existing format (JSON and TOML have been suggested, with JSON
being dismissed due to lack of comment support) which has quoting
conventions that supports postgres' object names and which can be used
to
define a full pg_dump configuration file syntax.

For B), C) and D) there is implicit consensus in the thread that we don't
need
to implement the full configuration file as of this patch, merely that it
*must* be possible to do so without having to paint ourselves out of a
corner.

At this point it seems to me that B) and C) has the broadest support. Can
the
C) option may represent the compromise between "simple" format for object
filtering and a more structured format for configuration? Are there other
options?

What should be a benefit of this variant?

Regards

Pavel

Show quoted text

Thoughts?

--
Daniel Gustafsson https://vmware.com/

[0] /messages/by-id/F6674FF0-5800-4AED-9DC7-13C475707241@yesql.se
[1]
/messages/by-id/CALAY4q9u30L7oGhbsfY3dPECQ8SrYa8YO=H-xOn5xWUeiEneeg@mail.gmail.com
[2] /messages/by-id/20201110200904.GU16415@tamriel.snowman.net
[3]
/messages/by-id/CAEZATCVKMG7+b+_5tNwrNZ-aNDBy3=FMRNea2bO9O4qGcEvSTg@mail.gmail.com
[4] /messages/by-id/502641.1606334432@sss.pgh.pa.us
[5] /messages/by-id/619671.1606406538@sss.pgh.pa.us
[6]
/messages/by-id/cb545d78-2dae-8d27-f062-822a07ca56cf@enterprisedb.com
[7] /messages/by-id/202107122259.n6o5uwb5erza@alvherre.pgsql
[8] /messages/by-id/3183720.1626131795@sss.pgh.pa.us

#101Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#99)
Re: proposal: possibility to read dumped table's name from file

pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 15 Sep 2021, at 19:31, Pavel Stehule <pavel.stehule@gmail.com> wrote:
po 13. 9. 2021 v 15:01 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se>> napsal:

One issue with this syntax is that the include keyword can be quite

misleading

as it's semantic interpretion of "include table t" can be different from
"--table=t". The former is less clear about the fact that it means

"exclude

all other tables than " then the latter. It can be solved with

documentation,

but I think that needs be to be made clearer.

I invite any documentation enhancing and fixing

Sure, that can be collabored on. This gist is though that IMO the
keywords in
the filter file aren't as clear on the sideeffects as the command line
params,
even though they are equal in functionality.

+       pg_log_error("invalid format of filter file \"%s\": %s",
+                                filename,
+                                message);
+
+       fprintf(stderr, "%d: %s\n", lineno, line);
Can't we just include the lineno in the error logging and skip dumping

the

offending line? Fast-forwarding the pointer to print the offending part

is

less useful than a context marker, and in some cases suboptimal. With

this

coding, if a pattern is omitted for example the below error message is

given:

pg_dump: error: invalid format of filter file "filter.txt": missing

object name

1:

The errormessage and the linenumber in the file should be enough for the

user

to figure out what to fix.

I did it like you proposed, but still, I think the content can be useful.

Not when there is no content in the error message, printing an empty
string for
a line number which isn't a blank line doesn't seem terribly helpful. If
we
know the error context is empty, printing a tailored error hint seems more
useful for the user.

More times you read dynamically generated files, or you read data from

stdin, and in complex environments it can be hard regenerate new content
for debugging.

That seems odd given that the arguments for this format has been that it's
likely to be handwritten.

If I create a table called "a\nb" and try to dump it I get an error in

parsing

the file. Isn't this supposed to work?
$ cat filter.txt
include table "a
b"
$ ./bin/pg_dump --filter=filter.txt
pg_dump: error: invalid format of filter file "filter.txt":

unexpected chars after object name

2:

probably there was some issue, because it should work. I tested a new

version and this is tested in new regress tests. Please, check

That seems to work, but I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Show quoted text

Did you consider implementing this in Bison to abstract some of the

messier

parsing logic?

Initially not, but now, when I am thinking about it, I don't think so

Bison helps. The syntax of the filter file is nicely linear. Now, the code
of the parser is a little bit larger than minimalistic, but it is due to
nicer error's messages. The raw implementation in Bison raised just "syntax
error" and positions. I did code refactoring, and now the scanning, parsing
and processing are divided into separated routines. Parsing related code
has 90 lines. In this case, I don't think using a parser grammar file can
carry any benefit. grammar is more readable, sure, but we need to include
bison, we need to handle errors, and if we want to raise more helpful
errors than just "syntax error", then the code will be longer.

I'm not so concerned by code size, but rather parsing of quotations etc and
being able to reason about it's correctness. IMHO that's easier done by
reading a defined grammar than parsing a handwritten parser.

Will do a closer review on the patch shortly.

--
Daniel Gustafsson https://vmware.com/

#102Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#101)
Re: proposal: possibility to read dumped table's name from file

On 17 Sep 2021, at 13:51, Pavel Stehule <pavel.stehule@gmail.com> wrote:
pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se>> napsal:

I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Sure, see below:

$ ./bin/psql filter
psql (15devel)
Type "help" for help.

filter=# create table "a""
filter"# ""b" (a integer);
CREATE TABLE
filter=# select relname from pg_class order by oid desc limit 1;
relname
---------
a" +
"b
(1 row)

filter=# ^D\q
$ ./bin/pg_dump -s filter
--
-- PostgreSQL database dump
--

-- Dumped from database version 15devel
-- Dumped by pg_dump version 15devel

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;

SET default_tablespace = '';

SET default_table_access_method = heap;

--
-- Name: a" "b; Type: TABLE; Schema: public; Owner: danielg
--

CREATE TABLE public."a""
""b" (
a integer
);

ALTER TABLE public."a""
""b" OWNER TO danielg;

--
-- PostgreSQL database dump complete
--

--
Daniel Gustafsson https://vmware.com/

#103Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#102)
Re: proposal: possibility to read dumped table's name from file

pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 17 Sep 2021, at 13:51, Pavel Stehule <pavel.stehule@gmail.com> wrote:
pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se>> napsal:

I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Sure, see below:

$ ./bin/psql filter
psql (15devel)
Type "help" for help.

I didn't ask on this

I asked if you can use -t and some for filtering this name

?

Show quoted text

filter=# create table "a""
filter"# ""b" (a integer);
CREATE TABLE
filter=# select relname from pg_class order by oid desc limit 1;
relname
---------
a" +
"b
(1 row)

filter=# ^D\q
$ ./bin/pg_dump -s filter
--
-- PostgreSQL database dump
--

-- Dumped from database version 15devel
-- Dumped by pg_dump version 15devel

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;

SET default_tablespace = '';

SET default_table_access_method = heap;

--
-- Name: a" "b; Type: TABLE; Schema: public; Owner: danielg
--

CREATE TABLE public."a""
""b" (
a integer
);

ALTER TABLE public."a""
""b" OWNER TO danielg;

--
-- PostgreSQL database dump complete
--

--
Daniel Gustafsson https://vmware.com/

#104Stephen Frost
sfrost@snowman.net
In reply to: Pavel Stehule (#103)
Re: proposal: possibility to read dumped table's name from file

Greetings,

On Fri, Sep 17, 2021 at 13:59 Pavel Stehule <pavel.stehule@gmail.com> wrote:

pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 17 Sep 2021, at 13:51, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se>> napsal:

I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Sure, see below:

$ ./bin/psql filter
psql (15devel)
Type "help" for help.

I didn't ask on this

I asked if you can use -t and some for filtering this name

?

For my part, at least, I don’t see that this particularly matters.. for a
new feature that’s being developed to allow users to export specific
tables, I would think we’d want to support any table names which can exist.

Thanks,

Stephen

#105Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#103)
Re: proposal: possibility to read dumped table's name from file

On 17 Sep 2021, at 13:59, Pavel Stehule <pavel.stehule@gmail.com> wrote:
pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se>> napsal:

On 17 Sep 2021, at 13:51, Pavel Stehule <pavel.stehule@gmail.com <mailto:pavel.stehule@gmail.com>> wrote:
pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se> <mailto:daniel@yesql.se <mailto:daniel@yesql.se>>> napsal:

I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Sure, see below:

$ ./bin/psql filter
psql (15devel)
Type "help" for help.

I didn't ask on this

I asked if you can use -t and some for filtering this name?

I didn't try as I don't see how that's relevant? Surely we're not limiting the
capabilities of a filtering file format based on the quoting semantics of a
shell?

--
Daniel Gustafsson https://vmware.com/

#106Stephen Frost
sfrost@snowman.net
In reply to: Daniel Gustafsson (#105)
Re: proposal: possibility to read dumped table's name from file

Greetings,

On Fri, Sep 17, 2021 at 14:07 Daniel Gustafsson <daniel@yesql.se> wrote:

On 17 Sep 2021, at 13:59, Pavel Stehule <pavel.stehule@gmail.com> wrote:
pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se>> napsal:

On 17 Sep 2021, at 13:51, Pavel Stehule <pavel.stehule@gmail.com

<mailto:pavel.stehule@gmail.com>> wrote:

pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se> <mailto:daniel@yesql.se <mailto:daniel@yesql.se>>>
napsal:

I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Sure, see below:

$ ./bin/psql filter
psql (15devel)
Type "help" for help.

I didn't ask on this

I asked if you can use -t and some for filtering this name?

I didn't try as I don't see how that's relevant? Surely we're not
limiting the
capabilities of a filtering file format based on the quoting semantics of a
shell?

Yeah, agreed. I would think that a DBA might specifically want to be able
to use a config file to get away from having to deal with shell quoting, in
fact…

Thanks,

Stephen

Show quoted text
#107Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#105)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

pá 17. 9. 2021 v 14:07 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 17 Sep 2021, at 13:59, Pavel Stehule <pavel.stehule@gmail.com> wrote:
pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se>> napsal:

On 17 Sep 2021, at 13:51, Pavel Stehule <pavel.stehule@gmail.com

<mailto:pavel.stehule@gmail.com>> wrote:

pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se> <mailto:daniel@yesql.se <mailto:daniel@yesql.se>>>
napsal:

I am unable to write a filter statement which can
handle this relname:

CREATE TABLE "a""
""b" (a integer);

Are you able to craft one for that?

I am not able to dump this directly in pg_dump. Is it possible?

Sure, see below:

$ ./bin/psql filter
psql (15devel)
Type "help" for help.

I didn't ask on this

I asked if you can use -t and some for filtering this name?

I didn't try as I don't see how that's relevant? Surely we're not
limiting the
capabilities of a filtering file format based on the quoting semantics of a
shell?

this patch just use existing functionality, that can be buggy too.

but I had a bug in this part - if I detect double double quotes on input I
have to send double quotes to output too.

It should be fixed in attached patch

[pavel@localhost pg_dump]$ echo 'include table "a""\n""b"' | ./pg_dump
--filter=-
--
-- PostgreSQL database dump
--

-- Dumped from database version 15devel
-- Dumped by pg_dump version 15devel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20210917.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20210917.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..1b74c0eadd 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,55 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file. Specify "-" to read from
+        stdin. Lines of this file must have the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included
+        or excluded, and the second keyword specifies the type of object
+        to be filtered:
+        <literal>table</literal> (table),
+        <literal>schema</literal> (schema),
+        <literal>foreign_data</literal> (foreign server),
+        <literal>data</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable1
+include foreign_data some_foreign_server
+exclude table mytable2
+</programlisting>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are ignored. The comment
+        (started by <literal>#</literal>) can be placed after filter too.
+        Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
+        tables, and both forms may be combined.  Note that there are no options
+        to exclude a specific foreign table or to include a specific table's
+        data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..0e8072a782 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -308,7 +310,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
-
+static void read_filters_from_file(char *filename, DumpOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -380,6 +382,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +616,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_filters_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1045,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +18988,346 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+typedef struct
+{
+	FILE	   *fp;
+	char	   *filename;
+	int			lineno;
+} FilterStateData;
+
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		fclose(fstate->fp);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isspace(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+static bool
+filter_is_keyword(const char *keyword, int size, const char *str)
+{
+	if (strlen(str) != size)
+		return false;
+
+	return pg_strncasecmp(keyword, str, size) == 0;
+}
+
+/*
+ * Sets objname to string with object identifier. The line variable holds string
+ * of last line with object identifier (object name). Returns pointer to first char
+ * after last char of object name.
+ */
+static char *
+filter_get_object_name(FilterStateData *fstate,
+					   StringInfo line,
+					   char *str,
+					   char **objname)
+{
+	/* skip white spaces */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name");
+
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (!pg_get_line_buf(fstate->fp, line))
+				{
+					if (ferror(fstate->fp))
+						fatal("could not read from file \"%s\": %m", fstate->filename);
+
+					exit_invalid_filter_format(fstate,"unexpected end of file");
+				}
+
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				str = line->data;
+				fstate->lineno += 1;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = quoted_name->data;
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* simple variant, read to end or to first space */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * Returns true, when one filter item was successfully read and parsed.
+ * When object name contains \n chars, then more than one line from input
+ * file can be processed. Returns false when EOF. Run exit on error.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * skip empty lines or lines when first noblank char is hash (comment)
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"include\" or \"exclude\")");
+
+			/* Now we expect sequence of two keywords */
+			if (filter_is_keyword(keyword, size, "include"))
+				*is_include = true;
+			else if (filter_is_keyword(keyword, size, "exclude"))
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (filter_is_keyword(keyword, size, "table"))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (filter_is_keyword(keyword, size, "schema"))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (filter_is_keyword(keyword, size, "foreign_data"))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (filter_is_keyword(keyword, size, "data"))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_object_name(fstate, &line, str, objname);
+
+			/*
+			 * check possible content after object identifier.
+			 * Allow comment started by hash.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+				   "unexpected chars after object name");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_fatal("could not read from file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+			fclose(fstate->fp);
+
+		exit_nicely(-1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_filters_from_file(char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		free(objname);
+	}
+
+	if (fstate.fp != stdin)
+		fclose(fstate.fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..505614f145
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,151 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 24;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"broken format check");
#108Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#99)
Re: proposal: possibility to read dumped table's name from file

Initially not, but now, when I am thinking about it, I don't think so

Bison helps. The syntax of the filter file is nicely linear. Now, the code
of the parser is a little bit larger than minimalistic, but it is due to
nicer error's messages. The raw implementation in Bison raised just "syntax
error" and positions. I did code refactoring, and now the scanning, parsing
and processing are divided into separated routines. Parsing related code
has 90 lines. In this case, I don't think using a parser grammar file can
carry any benefit. grammar is more readable, sure, but we need to include
bison, we need to handle errors, and if we want to raise more helpful
errors than just "syntax error", then the code will be longer.

I'm not so concerned by code size, but rather parsing of quotations etc and
being able to reason about it's correctness. IMHO that's easier done by
reading a defined grammar than parsing a handwritten parser.

In this case the complex part is not a parser, but the scanner is complex
and writing this in flex is not too easy. I wrote so the grammar file can
be more readable, but the usual error from Bison is "syntax error" and
position, so it does not win from the user perspective. When a parser is
not linear, then a generated parser can help a lot, but using it at this
moment is premature.

Show quoted text

Will do a closer review on the patch shortly.

--
Daniel Gustafsson https://vmware.com/

#109Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#108)
Re: proposal: possibility to read dumped table's name from file

Will do a closer review on the patch shortly.

Had a read through, and tested, the latest posted version today:

+    Read objects filters from the specified file. Specify "-" to read from
+    stdin. Lines of this file must have the following format:
I think this should be <filename>-</filename> and <literal>STDIN</literal> to
match the rest of the docs.
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable1
+include foreign_data some_foreign_server
+exclude table mytable2
+</programlisting>
+       </para>
This example is highlighting the issue I've previously raised with the UX/doc
of this feature.  The "exclude table mytable2" is totally pointless in the
above since the exact match of "mytable1" will remove all other objects.  What
we should be doing instead is use the pattern matching aspect along the lines
of the below:

include table mytable*
exclude table mytable2

+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or foreign
This should refer to the actual options by name to make it clear which we are
talking about.
+       printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+                        "                               from the filter file\n"));
Before we settle on --filter I think we need to conclude whether this file is
intended to be included from a config file, or used on it's own.  If we gow tih
the former then we might not want a separate option for just --filter.

+ if (filter_is_keyword(keyword, size, "include"))
I would prefer if this function call was replaced by just the pg_strcasecmp()
call in filter_is_keyword() and the strlen optimization there removed. The is
not a hot-path, we can afford the string comparison in case of errors. Having
the string comparison done inline here will improve readability saving the
reading from jumping to another function to see what it does.

+ initStringInfo(&line);
Why is this using a StringInfo rather than a PQExpBuffer as the rest of pg_dump
does?

+typedef struct
I think these should be at the top of the file with the other typedefs.

When testing strange object names, I was unable to express this name in the filter file:

$ ./bin/psql
psql (15devel)
Type "help" for help.

danielg=# create table "
danielg"# t
danielg"# t
danielg"# " (a integer);
CREATE TABLE
danielg=# select relname from pg_class order by oid desc limit 1;
relname
---------
+
t +
t +

(1 row)

--
Daniel Gustafsson https://vmware.com/

#110Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#109)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

po 20. 9. 2021 v 14:10 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

Will do a closer review on the patch shortly.

Had a read through, and tested, the latest posted version today:

+    Read objects filters from the specified file. Specify "-" to read from
+    stdin. Lines of this file must have the following format:
I think this should be <filename>-</filename> and <literal>STDIN</literal>
to
match the rest of the docs.
+       <para>
+        With the following filter file, the dump would include table
+        <literal>mytable1</literal> and data from foreign tables of
+        <literal>some_foreign_server</literal> foreign server, but
exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable1
+include foreign_data some_foreign_server
+exclude table mytable2
+</programlisting>
+       </para>
This example is highlighting the issue I've previously raised with the
UX/doc
of this feature.  The "exclude table mytable2" is totally pointless in the
above since the exact match of "mytable1" will remove all other objects.
What
we should be doing instead is use the pattern matching aspect along the
lines
of the below:

include table mytable*
exclude table mytable2

+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables, schemas, table data, or
foreign
This should refer to the actual options by name to make it clear which we
are
talking about.

fixed

+       printf(_("  --filter=FILENAME            dump objects and data
based on the filter expressions\n"
+                        "                               from the filter
file\n"));
Before we settle on --filter I think we need to conclude whether this file
is
intended to be included from a config file, or used on it's own.  If we
gow tih
the former then we might not want a separate option for just --filter.

I prefer to separate two files. Although there is some intersection, I
think it is good to have two simple separate files for two really different
tasks.
It does filtering, and it should be controlled by option "--filter". When
the implementation will be changed, then this option can be changed too.
Filtering is just a pg_dump related feature. Revision of client application
configuration is a much more generic task, and if we mix it to one, we can
be
in a trap. It can be hard to find one good format for large script
generated content, and possibly hand written structured content. For
practical
reasons it can be good to have two files too. Filters and configurations
can have different life cycles.

+ if (filter_is_keyword(keyword, size, "include"))
I would prefer if this function call was replaced by just the
pg_strcasecmp()
call in filter_is_keyword() and the strlen optimization there removed.
The is
not a hot-path, we can afford the string comparison in case of errors.
Having
the string comparison done inline here will improve readability saving the
reading from jumping to another function to see what it does.

I agree that this is not a hot-path, just I don't feel well if I need to
make a zero end string just for comparison pg_strcasecmp. Current design
reduces malloc/free cycles. It is used in more places, when Postgres parses
strings - SQL parser, plpgsql parser. I am not sure about the benefits and
costs - pg_strcasecmp can be more readable, but for any keyword I have to
call pstrdup and pfree. Is it necessary? My opinion in this part is not too
strong - it is a minor issue, maybe I have a little bit different feelings
about benefits and costs in this specific case, and if you really think the
benefits of rewriting are higher, I'll do it.

+ initStringInfo(&line);
Why is this using a StringInfo rather than a PQExpBuffer as the rest of
pg_dump
does?

The StringInfo is used because I use the pg_get_line_buf function, and this
function uses this API.

+typedef struct
I think these should be at the top of the file with the other typedefs.

done

When testing strange object names, I was unable to express this name in
the filter file:

$ ./bin/psql
psql (15devel)
Type "help" for help.

danielg=# create table "
danielg"# t
danielg"# t
danielg"# " (a integer);
CREATE TABLE
danielg=# select relname from pg_class order by oid desc limit 1;
relname
---------
+
t +
t +

(1 row)

Good catch - I had badly placed pg_strip_crlf function, fixed and regress
tests enhanced

Please check assigned patch

Regards

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20210921.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20210921.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..d692f4230e 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,55 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Read objects filters from the specified file. Specify <filename>-</filename> to read from
+        <literal>STDIN</literal>. Lines of this file must have the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included
+        or excluded, and the second keyword specifies the type of object
+        to be filtered:
+        <literal>table</literal> (table),
+        <literal>schema</literal> (schema),
+        <literal>foreign_data</literal> (foreign server),
+        <literal>data</literal> (table data).
+       </para>
+
+       <para>
+        With the following filter file, the dump would include tables with 
+        name starting by <literal>mytable</literal>, but exclude data
+        from table <literal>mytable2</literal>.
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are ignored. The comment
+        (started by <literal>#</literal>) can be placed after filter too.
+        Empty lines are ignored too.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables (<option>-t</option> or
+        <option>--table</option>), schemas (<option>-n</option> or
+        <option>--schema</option>), table data (<option>--exclude-table-data</option>),
+        or foreign tables (<option>--include-foreign-data</option>.
+        Isn't possible to exclude a specific foreign table and isn't possible
+        to include a specific table's data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..4da5e6c771 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -90,6 +92,28 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream.
+ */
+typedef struct
+{
+	FILE	   *fp;
+	char	   *filename;
+	int			lineno;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -308,7 +332,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
-
+static void read_filters_from_file(char *filename, DumpOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -380,6 +404,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +638,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_filters_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1067,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               from the filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +19010,330 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		fclose(fstate->fp);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(-1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isspace(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+static bool
+filter_is_keyword(const char *keyword, int size, const char *str)
+{
+	if (strlen(str) != size)
+		return false;
+
+	return pg_strncasecmp(keyword, str, size) == 0;
+}
+
+/*
+ * Sets objname to string with object identifier. The line variable holds string
+ * of last line with object identifier (object name). Returns pointer to first char
+ * after last char of object name.
+ */
+static char *
+filter_get_object_name(FilterStateData *fstate,
+					   StringInfo line,
+					   char *str,
+					   char **objname)
+{
+	/* skip white spaces */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name");
+
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (!pg_get_line_buf(fstate->fp, line))
+				{
+					if (ferror(fstate->fp))
+						fatal("could not read from file \"%s\": %m", fstate->filename);
+
+					exit_invalid_filter_format(fstate,"unexpected end of file");
+				}
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno += 1;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = quoted_name->data;
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* simple variant, read to end or to first space */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * Returns true, when one filter item was successfully read and parsed.
+ * When object name contains \n chars, then more than one line from input
+ * file can be processed. Returns false when EOF. Run exit on error.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * skip empty lines or lines when first noblank char is hash (comment)
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"include\" or \"exclude\")");
+
+			/* Now we expect sequence of two keywords */
+			if (filter_is_keyword(keyword, size, "include"))
+				*is_include = true;
+			else if (filter_is_keyword(keyword, size, "exclude"))
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (filter_is_keyword(keyword, size, "table"))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (filter_is_keyword(keyword, size, "schema"))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (filter_is_keyword(keyword, size, "foreign_data"))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (filter_is_keyword(keyword, size, "data"))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_object_name(fstate, &line, str, objname);
+
+			/*
+			 * check possible content after object identifier.
+			 * Allow comment started by hash.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+				   "unexpected chars after object name");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_fatal("could not read from file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+			fclose(fstate->fp);
+
+		exit_nicely(-1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_filters_from_file(char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		free(objname);
+	}
+
+	if (fstate.fp != stdin)
+		fclose(fstate.fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..85b84db855
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,185 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 28;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include table \"
+t
+t
+\"";
+
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include table \"\\nt\\nt\\n\"";
+
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+#########################################
+# For test of +f option we need created foreign server or accept
+# fail and check error
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"broken format check");
#111Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#110)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 21 Sep 2021, at 08:50, Pavel Stehule <pavel.stehule@gmail.com> wrote:
po 20. 9. 2021 v 14:10 odesílatel Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se>> napsal:

+       printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+                        "                               from the filter file\n"));
Before we settle on --filter I think we need to conclude whether this file is
intended to be included from a config file, or used on it's own.  If we gow tih
the former then we might not want a separate option for just --filter.

I prefer to separate two files. Although there is some intersection, I think it is good to have two simple separate files for two really different tasks.
It does filtering, and it should be controlled by option "--filter". When the implementation will be changed, then this option can be changed too.
Filtering is just a pg_dump related feature. Revision of client application configuration is a much more generic task, and if we mix it to one, we can be
in a trap. It can be hard to find one good format for large script generated content, and possibly hand written structured content. For practical
reasons it can be good to have two files too. Filters and configurations can have different life cycles.

I'm not convinced that we can/should change or remove a commandline parameter
in a coming version when there might be scripts expecting it to work in a
specific way. Having a --filter as well as a --config where the configfile can
refer to the filterfile also passed via --filter sounds like problem waiting to
happen, so I think we need to settle how we want to interact with this file
before anything goes in.

Any thoughts from those in the thread who have had strong opinions on config
files etc?

+ if (filter_is_keyword(keyword, size, "include"))
I would prefer if this function call was replaced by just the pg_strcasecmp()
call in filter_is_keyword() and the strlen optimization there removed. The is
not a hot-path, we can afford the string comparison in case of errors. Having
the string comparison done inline here will improve readability saving the
reading from jumping to another function to see what it does.

I agree that this is not a hot-path, just I don't feel well if I need to make a zero end string just for comparison pg_strcasecmp. Current design reduces malloc/free cycles. It is used in more places, when Postgres parses strings - SQL parser, plpgsql parser. I am not sure about the benefits and costs - pg_strcasecmp can be more readable, but for any keyword I have to call pstrdup and pfree. Is it necessary? My opinion in this part is not too strong - it is a minor issue, maybe I have a little bit different feelings about benefits and costs in this specific case, and if you really think the benefits of rewriting are higher, I'll do it

Sorry, I typoed my response. What I meant was to move the pg_strncasecmp call
inline and not do the strlen check, to save readers from jumping around. So
basically end up with the below in read_filter_item():

+	/* Now we expect sequence of two keywords */
+	if (pg_strncasecmp(keyword, "include", size) == 0)
+		*is_include = true;

+ initStringInfo(&line);
Why is this using a StringInfo rather than a PQExpBuffer as the rest of pg_dump
does?

The StringInfo is used because I use the pg_get_line_buf function, and this function uses this API.

Ah, of course.

A few other comments from another pass over this:

+ exit_nicely(-1);
Why -1? pg_dump (and all other binaries) exits with 1 on IMO even more serious
errors so I think this should use 1 as well.

+	if (!pg_get_line_buf(fstate->fp, line))
+	{
+		if (ferror(fstate->fp))
+			fatal("could not read from file \"%s\": %m", fstate->filename);
+
+		exit_invalid_filter_format(fstate,"unexpected end of file");
+	}
In the ferror() case this codepath isn't running fclose() on the file pointer
(unless stdin) which we do elsewhere, so this should use pg_log_error and
exit_nicely instead.

+ pg_log_fatal("could not read from file \"%s\": %m", fstate->filename);
Based on how other errors are treated in pg_dump I think this should be
downgraded to a pg_log_error.

The above comments are fixed in the attached, as well as a pass over the docs
and extended tests to actually test matching a foreign server. What do think
about this version? I'm still not convinced that there aren't more quoting
bugs in the parser, but I've left that intact for now.

--
Daniel Gustafsson https://vmware.com/

Attachments:

0001-Implement-filter-for-pg_dump.patchapplication/octet-stream; name=0001-Implement-filter-for-pg_dump.patch; x-unix-mode=0644Download
From 7fcc71c322bc700119c2fafa15884bad2a43f0ff Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Tue, 21 Sep 2021 14:26:35 +0200
Subject: [PATCH] Implement --filter for pg_dump

Author: Pavel Stehule
---
 doc/src/sgml/ref/pg_dump.sgml               |  61 ++++
 src/bin/pg_dump/pg_dump.c                   | 356 ++++++++++++++++++++
 src/bin/pg_dump/t/004_pg_dump_filterfile.pl | 198 +++++++++++
 3 files changed, 615 insertions(+)
 create mode 100644 src/bin/pg_dump/t/004_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..6e9a56f68c 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,67 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+      Filtering rules for which objects to dump are read from the specified
+      file.  Specify <filename>-</filename> to read from
+    <literal>STDIN</literal>.  The file has the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included or
+        excluded, and the second keyword specifies the type of object to be
+        filtered:
+        <itemizedlist>
+         <listitem>
+           <para><literal>table</literal>: table</para>
+         </listitem>
+     <listitem>
+       <para><literal>schema</literal>: schema</para>
+     </listitem>
+     <listitem>
+           <para><literal>foreign_data</literal>: foreign server</para>
+     </listitem>
+     <listitem>
+           <para><literal>data</literal>: table data</para>
+     </listitem>
+    </itemizedlist>
+       </para>
+
+       <para>
+        Example:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+        With the this filter file, the dump would include all tables with 
+        name starting by <literal>mytable</literal>, but exclude table
+        <literal>mytable2</literal>.
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables (<option>-t</option> or
+        <option>--table</option>), schemas (<option>-n</option> or
+        <option>--schema</option>), table data (<option>--exclude-table-data</option>),
+        or foreign tables (<option>--include-foreign-data</option>).
+        It isn't possible to exclude a specific foreign table or
+        to include a specific table's data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..45b1626a85 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -90,6 +92,28 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream.
+ */
+typedef struct
+{
+	FILE	   *fp;
+	char	   *filename;
+	int			lineno;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -308,6 +332,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_filters_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -380,6 +405,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +639,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_filters_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1068,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +19011,327 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		fclose(fstate->fp);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isspace(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * Sets objname to string with object identifier. The line variable holds
+ * string of last line with object identifier (object name). Returns pointer to
+ * first char after object name.
+ */
+static char *
+filter_get_object_name(FilterStateData *fstate,
+					   StringInfo line,
+					   char *str,
+					   char **objname)
+{
+	/* skip white spaces */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name");
+
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (!pg_get_line_buf(fstate->fp, line))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from file \"%s\": %m", fstate->filename);
+						if (fstate->fp != stdin)
+							fclose(fstate->fp);
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate,"unexpected end of file");
+				}
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno += 1;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = quoted_name->data;
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* simple variant, read to end or to first space */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * Returns true, when one filter item was successfully read and parsed.
+ * When object name contains \n chars, then more than one line from input
+ * file can be processed. Returns false when EOF. Run exit on error.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * skip empty lines or lines when first noblank char is hash (comment)
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"include\" or \"exclude\")");
+
+			/* Now we expect sequence of two keywords */
+			if (pg_strncasecmp(keyword, "include", size) == 0)
+				*is_include = true;
+			else if (pg_strncasecmp(keyword, "exclude", size) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (pg_strncasecmp(keyword, "table", size) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (pg_strncasecmp(keyword, "schema", size) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (pg_strncasecmp(keyword, "foreign_data", size) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (pg_strncasecmp(keyword, "data", size) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_object_name(fstate, &line, str, objname);
+
+			/*
+			 * check possible content after object identifier.
+			 * Allow comment started by hash.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+				   "unexpected chars after object name");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+			fclose(fstate->fp);
+
+		exit_nicely(1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_filters_from_file(char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		free(objname);
+	}
+
+	if (fstate.fp != stdin)
+		fclose(fstate.fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..ccb28c6b8c
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,198 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 29;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres', 'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+#########################################
+# Test foreign_data 
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummy*\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump foreign_data with filter");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"broken format check");
-- 
2.24.3 (Apple Git-128)

#112Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#111)
Re: proposal: possibility to read dumped table's name from file

út 21. 9. 2021 v 14:37 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 21 Sep 2021, at 08:50, Pavel Stehule <pavel.stehule@gmail.com> wrote:
po 20. 9. 2021 v 14:10 odesílatel Daniel Gustafsson <daniel@yesql.se

<mailto:daniel@yesql.se>> napsal:

+ printf(_(" --filter=FILENAME dump objects and data

based on the filter expressions\n"

+ " from the filter

file\n"));

Before we settle on --filter I think we need to conclude whether this

file is

intended to be included from a config file, or used on it's own. If we

gow tih

the former then we might not want a separate option for just --filter.

I prefer to separate two files. Although there is some intersection, I

think it is good to have two simple separate files for two really different
tasks.

It does filtering, and it should be controlled by option "--filter".

When the implementation will be changed, then this option can be changed
too.

Filtering is just a pg_dump related feature. Revision of client

application configuration is a much more generic task, and if we mix it to
one, we can be

in a trap. It can be hard to find one good format for large script

generated content, and possibly hand written structured content. For
practical

reasons it can be good to have two files too. Filters and configurations

can have different life cycles.

I'm not convinced that we can/should change or remove a commandline
parameter
in a coming version when there might be scripts expecting it to work in a
specific way. Having a --filter as well as a --config where the
configfile can
refer to the filterfile also passed via --filter sounds like problem
waiting to
happen, so I think we need to settle how we want to interact with this file
before anything goes in.

Any thoughts from those in the thread who have had strong opinions on
config
files etc?

+ if (filter_is_keyword(keyword, size, "include"))
I would prefer if this function call was replaced by just the

pg_strcasecmp()

call in filter_is_keyword() and the strlen optimization there removed.

The is

not a hot-path, we can afford the string comparison in case of errors.

Having

the string comparison done inline here will improve readability saving

the

reading from jumping to another function to see what it does.

I agree that this is not a hot-path, just I don't feel well if I need to

make a zero end string just for comparison pg_strcasecmp. Current design
reduces malloc/free cycles. It is used in more places, when Postgres parses
strings - SQL parser, plpgsql parser. I am not sure about the benefits and
costs - pg_strcasecmp can be more readable, but for any keyword I have to
call pstrdup and pfree. Is it necessary? My opinion in this part is not too
strong - it is a minor issue, maybe I have a little bit different feelings
about benefits and costs in this specific case, and if you really think the
benefits of rewriting are higher, I'll do it

Sorry, I typoed my response. What I meant was to move the pg_strncasecmp
call
inline and not do the strlen check, to save readers from jumping around.
So
basically end up with the below in read_filter_item():

+       /* Now we expect sequence of two keywords */
+       if (pg_strncasecmp(keyword, "include", size) == 0)
+               *is_include = true;

I don't think so it is safe (strict). Only pg_strncasecmp(..) is true for
keywords "includex", "includedsss", ... You should to compare the size

Regards

Pavel

Show quoted text
#113Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Daniel Gustafsson (#111)
Re: proposal: possibility to read dumped table's name from file

I definitely agree that we should have two files, one for config and
another one for filter, since their purposes are orthogonal and their
formats are likely different; trying to cram the filter specification in
the config file seems unfriendly because it'd force users to write the
filter in whatever alien grammar used for the config file. Also, this
would make it easier to use a single config file with a bunch of
different filter files.

On 2021-Sep-21, Daniel Gustafsson wrote:

I'm not convinced that we can/should change or remove a commandline parameter
in a coming version when there might be scripts expecting it to work in a
specific way. Having a --filter as well as a --config where the configfile can
refer to the filterfile also passed via --filter sounds like problem waiting to
happen, so I think we need to settle how we want to interact with this file
before anything goes in.

I think both the filter and the hypothetical config file are going to
interact (be redundant) with almost all already existing switches, and
there's no need to talk about removing anything (e.g., nobody would
argue for the removal of "-t" even though that's redundant with the
filter file).

I see no problem with the config file specifying a filter file.

AFAICS if the config file specifies a filter and the user also specifies
a filter in the command line, we have two easy options: raise an error
about the redundant option, or have the command line option supersede
the one in the config file. The latter strikes me as the more useful
behavior, and it's in line with what other tools do in similar cases, so
that's what I propose doing.

(There might be less easy options too, such as somehow combining the two
filters, but offhand I don't see any reason why this is real-world
useful, so I don't propose doing that.)

--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"How amazing is that? I call it a night and come back to find that a bug has
been identified and patched while I sleep." (Robert Davidson)
http://archives.postgresql.org/pgsql-sql/2006-03/msg00378.php

#114Tomas Vondra
tomas.vondra@enterprisedb.com
In reply to: Alvaro Herrera (#113)
Re: proposal: possibility to read dumped table's name from file

On 9/21/21 3:28 PM, Alvaro Herrera wrote:

I definitely agree that we should have two files, one for config and
another one for filter, since their purposes are orthogonal and their
formats are likely different; trying to cram the filter specification in
the config file seems unfriendly because it'd force users to write the
filter in whatever alien grammar used for the config file. Also, this
would make it easier to use a single config file with a bunch of
different filter files.

+1, that is pretty much excatly what I argued for not too long ago.

On 2021-Sep-21, Daniel Gustafsson wrote:

I'm not convinced that we can/should change or remove a commandline parameter
in a coming version when there might be scripts expecting it to work in a
specific way. Having a --filter as well as a --config where the configfile can
refer to the filterfile also passed via --filter sounds like problem waiting to
happen, so I think we need to settle how we want to interact with this file
before anything goes in.

I think both the filter and the hypothetical config file are going to
interact (be redundant) with almost all already existing switches, and
there's no need to talk about removing anything (e.g., nobody would
argue for the removal of "-t" even though that's redundant with the
filter file).

I see no problem with the config file specifying a filter file.

AFAICS if the config file specifies a filter and the user also specifies
a filter in the command line, we have two easy options: raise an error
about the redundant option, or have the command line option supersede
the one in the config file. The latter strikes me as the more useful
behavior, and it's in line with what other tools do in similar cases, so
that's what I propose doing.

(There might be less easy options too, such as somehow combining the two
filters, but offhand I don't see any reason why this is real-world
useful, so I don't propose doing that.)

Well, I think we already have to do decisions like that, because you can
do e.g. this:

pg_dump -T t -t t

So we already do combine the switches, and we do this:

When both -t and -T are given, the behavior is to dump just the
tables that match at least one -t switch but no -T switches. If -T
appears without -t, then tables matching -T are excluded from what
is otherwise a normal dump.

That seems fairly reasonable, and I don't see why not to use the same
logic for combining patterns no matter where we got them (filter file,
command-line option, etc.).

Just combine everything, and then check if there's any "exclude" rule.
If yes, we're done - exclude. If not, check if there's "include" rule.
If not, still exclude. Otherwise include.

Seems reasonable and consistent to me, and I don't see why not to allow
multiple --filter parameters.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#115Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#111)
Re: proposal: possibility to read dumped table's name from file

Hi

The above comments are fixed in the attached, as well as a pass over the
docs
and extended tests to actually test matching a foreign server. What do
think
about this version? I'm still not convinced that there aren't more quoting
bugs in the parser, but I've left that intact for now.

The problematic points are double quotes and new line char. Any other is
just in sequence of bytes.

I have just one note to your patch. When you use pg_strncasecmp, then you
have to check the size too

char *xxx = "incl";
int xxx_size = 4;

elog(NOTICE, ">>>>%d<<<<",
pg_strncasecmp(xxx, "include", xxx_size) == 0);

result is NOTICE: >>>>1<<<<

"incl" is not keyword "include"

Regards

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

#116Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#111)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

The above comments are fixed in the attached, as well as a pass over the
docs
and extended tests to actually test matching a foreign server. What do
think
about this version? I'm still not convinced that there aren't more quoting
bugs in the parser, but I've left that intact for now.

This patch is based on the version that you sent 21.9. Just I modified
string comparison in keyword detection. If we don't allow support
abbreviations of keywords (and I dislike it), then the check of size is
necessary. Any other is without change.

Regards

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20210924.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20210924.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..6e9a56f68c 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,67 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+      Filtering rules for which objects to dump are read from the specified
+      file.  Specify <filename>-</filename> to read from
+    <literal>STDIN</literal>.  The file has the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the object is to be included or
+        excluded, and the second keyword specifies the type of object to be
+        filtered:
+        <itemizedlist>
+         <listitem>
+           <para><literal>table</literal>: table</para>
+         </listitem>
+     <listitem>
+       <para><literal>schema</literal>: schema</para>
+     </listitem>
+     <listitem>
+           <para><literal>foreign_data</literal>: foreign server</para>
+     </listitem>
+     <listitem>
+           <para><literal>data</literal>: table data</para>
+     </listitem>
+    </itemizedlist>
+       </para>
+
+       <para>
+        Example:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+        With the this filter file, the dump would include all tables with 
+        name starting by <literal>mytable</literal>, but exclude table
+        <literal>mytable2</literal>.
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables (<option>-t</option> or
+        <option>--table</option>), schemas (<option>-n</option> or
+        <option>--schema</option>), table data (<option>--exclude-table-data</option>),
+        or foreign tables (<option>--include-foreign-data</option>).
+        It isn't possible to exclude a specific foreign table or
+        to include a specific table's data.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..ec979b9f6f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -90,6 +92,28 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream.
+ */
+typedef struct
+{
+	FILE	   *fp;
+	char	   *filename;
+	int			lineno;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -308,6 +332,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_filters_from_file(char *filename, DumpOptions *dopt);
 
 
 int
@@ -380,6 +405,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +639,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* filter implementation */
+				read_filters_from_file(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1068,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +19011,327 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * Print error message and exit.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		fclose(fstate->fp);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+/*
+ * Search keyword (can contains only ascii alphabetic characters) on line.
+ * Returns NULL, when the line is empty or first char is not alpha
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* skip initial white spaces */
+	while (isspace(*ptr))
+		ptr += 1;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr += 1;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * Sets objname to string with object identifier. The line variable holds
+ * string of last line with object identifier (object name). Returns pointer to
+ * first char after object name.
+ */
+static char *
+filter_get_object_name(FilterStateData *fstate,
+					   StringInfo line,
+					   char *str,
+					   char **objname)
+{
+	/* skip white spaces */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name");
+
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (!pg_get_line_buf(fstate->fp, line))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from file \"%s\": %m", fstate->filename);
+						if (fstate->fp != stdin)
+							fclose(fstate->fp);
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate,"unexpected end of file");
+				}
+
+				str = line->data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno += 1;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = quoted_name->data;
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* simple variant, read to end or to first space */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * Returns true, when one filter item was successfully read and parsed.
+ * When object name contains \n chars, then more than one line from input
+ * file can be processed. Returns false when EOF. Run exit on error.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno += 1;
+
+		(void) pg_strip_crlf(str);
+
+		/* skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * skip empty lines or lines when first noblank char is hash (comment)
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"include\" or \"exclude\")");
+
+			/* Now we expect sequence of two keywords */
+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no keyword found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid keyword (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_object_name(fstate, &line, str, objname);
+
+			/*
+			 * check possible content after object identifier.
+			 * Allow comment started by hash.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+				   "unexpected chars after object name");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+			fclose(fstate->fp);
+
+		exit_nicely(1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * Read dumped object specification from file
+ */
+static void
+read_filters_from_file(char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	/* use "-" as symbol for stdin */
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open the input file \"%s\": %m",
+				  filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		free(objname);
+	}
+
+	if (fstate.fp != stdin)
+		fclose(fstate.fp);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..adc5675e40
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,198 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 29;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres', 'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+my ($cmd, $stdout, $stderr, $result);
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public.table_three/m, "table three not dumped");
+ok($dump !~ qr/^COPY public.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public.table_two/m, "content of table two is included");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public.table_three/m, "dumped table three");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummy*\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump foreign_data with filter");
+
+#########################################
+# Test broken input format
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid keyword/,
+	"broken format check");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"broken format check");
#117Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#116)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

I took another pass over this today and touched up the documentation (docs and
code) as well as tweaked the code a bit here and there to both make it fit the
pg_dump style better and to clean up a few small things. I've also added a set
of additional tests to cover more of the functionality.

I'm still not happy with the docs, I need to take another look there and see if
I make them more readable but otherwise I don't think there are any open issues
with this.

As has been discussed upthread, this format strikes a compromise wrt simplicity
and doesn't preclude adding a more structured config file in the future should
we want that. I think this takes care of most comments and opinions made in
this thread.

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20211001.patchapplication/octet-stream; name=pg_dump-filteropt-20211001.patch; x-unix-mode=0644Download
From 6d2edf9144685436702829acd60b802fb52c3306 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Fri, 1 Oct 2021 15:03:15 +0200
Subject: [PATCH v2] Add --filter option for reading object patterns from file

This adds a --filter=FILENAME option to pg_dump for specifying a set of
inclusion/exclusion patterns in a file. In situations where the wanted
filterset exceeds the commandline capabilities, this can be a handy way
to still be able to filter objects. The format of the file is line based
with each pattern on its own line:

<command> <object_type> <pattern>

When object names contain whitespace the pattern must be quoted.

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 doc/src/sgml/ref/pg_dump.sgml               |  63 +++
 src/bin/pg_dump/pg_dump.c                   | 419 ++++++++++++++++++++
 src/bin/pg_dump/t/004_pg_dump_filterfile.pl | 294 ++++++++++++++
 3 files changed, 776 insertions(+)
 create mode 100644 src/bin/pg_dump/t/004_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..2c266f60aa 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,69 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Filtering rules for which objects to dump are read from the specified
+        file.  Specify <filename>-</filename> to read from
+        <literal>STDIN</literal>.  The file has the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+		are to be included or excluded. The second keyword specifies the type
+		of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para><literal>table</literal>: table</para>
+         </listitem>
+         <listitem>
+          <para><literal>schema</literal>: schema</para>
+         </listitem>
+         <listitem>
+          <para><literal>foreign_data</literal>: foreign server</para>
+         </listitem>
+         <listitem>
+          <para><literal>data</literal>: table data</para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Example:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+        With this filter file, the dump would include all tables with 
+        name starting by <literal>mytable</literal>, except for table
+        <literal>mytable2</literal> which is explicitly excluded.
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored.
+       </para>
+
+       <para>
+        The <option>--filter</option> option works just like the other
+        options to include or exclude tables (<option>-t</option> or
+        <option>--table</option>), schemas (<option>-n</option> or
+        <option>--schema</option>), table data (<option>--exclude-table-data</option>),
+        or foreign tables (<option>--include-foreign-data</option>).
+        It isn't possible to exclude a specific foreign table or
+        to include a specific table's data.  The <option>--filter</option>
+        option can be specified in conjunction with these options, and can
+        also be specified more than once if there are multiple filter files.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..f074c363b7 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -90,6 +92,28 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+} FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -308,6 +332,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -380,6 +405,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +639,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1068,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +19011,390 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * exit_invalid_filter_format - Emit error message, close the file and exit
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		if (fclose(fstate->fp) != 0)
+			fatal("could not close filter file \"%s\": %m", fstate->filename);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic characters) in
+ * the passed in line buffer.  Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char	   *ptr = *line;
+	const char	   *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern.  Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+					   char *str,
+					   char **objname)
+{
+	StringInfoData line;
+
+	/* We only allocate the buffer if we need it */
+	line.data = NULL;
+
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name pattern");
+
+	/*
+	 * If the object name pattern has been quoted we must take care parse out
+	 * the entire quoted pattern, which may contain whitespace and can span
+	 * over many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer		quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (line.data == NULL)
+					initStringInfo(&line);
+
+				if (!pg_get_line_buf(fstate->fp, &line))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						if (fstate->fp != stdin)
+						{
+							if (fclose(fstate->fp) != 0)
+								fatal("could not close filter file \"%s\": %m",
+									  fstate->filename);
+						}
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate, "unexpected end of file");
+				}
+
+				str = line.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	if (line.data != NULL)
+		pg_free(line.data);
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a row
+ * based format a pattern may span more than one line due to how object names
+ * can be constructed.  The expected format of the filter file is:
+ * 
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of: "table",
+ * "schema", "foreign_data" or "data". The pattern is either simple without any
+ * whitespace, or properly quoted in case there is whitespace in the object
+ * name. The pattern handling follows the same rules as other object include
+ * and exclude functions; it can use wildcards Returns true, when one filter
+ * item was successfully read and parsed.  When object name contains \n chars,
+ * then more than one line from input file can be processed. Returns false when
+ * the filter file reaches EOF.  In case of errors, the function wont return
+ * but will exit with an appropriate error message.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData		line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char	   *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no filtercommand found (expected \"include\" or \"exclude\")");
+
+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid filtercommand (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+				   "no object type found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+				   "invalid object type (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_pattern(fstate, str, objname);
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the user
+			 * needed to quote the object name so exit with an invalid format
+			 * error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+										   "unexpected extra data after pattern");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+		{
+			if (fclose(fstate->fp) != 0)
+				fatal("could not close filter file \"%s\": %m", fstate->filename);
+		}
+
+		exit_nicely(1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+					   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+					   "exclude filter is not allowed for this type of object");
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..ec4ee9d8a7
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,294 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 45;
+
+my $tempdir       = TestLib::tempdir;
+my $inputfile;
+
+my $node = PostgresNode->new('main');
+my $port = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres', 'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres', "INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres', "INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, '-f', $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m, "tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m, "table three_one not dumped");
+ok($dump !~ qr/^COPY public\.table_one/m, "content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m, "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m, "dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+	or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", "--filter=$tempdir/inputfile2.txt", 'postgres' ],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, '-f', $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms, "dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, '-f', $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[ "pg_dump", '-p', $port, '-f', $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, '-f', $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid filtercommand/,
+	"invalid syntax: incorrect filtercommand");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/invalid object type/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data");
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+	or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[ "pg_dump", '-p', $port, "-f", $plainfile, "--filter=$tempdir/inputfile.txt", 'postgres' ],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
-- 
2.24.3 (Apple Git-128)

#118Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#117)
Re: proposal: possibility to read dumped table's name from file

pá 1. 10. 2021 v 15:19 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

I took another pass over this today and touched up the documentation (docs
and
code) as well as tweaked the code a bit here and there to both make it fit
the
pg_dump style better and to clean up a few small things. I've also added
a set
of additional tests to cover more of the functionality.

I'm still not happy with the docs, I need to take another look there and
see if
I make them more readable but otherwise I don't think there are any open
issues
with this.

As has been discussed upthread, this format strikes a compromise wrt
simplicity
and doesn't preclude adding a more structured config file in the future
should
we want that. I think this takes care of most comments and opinions made
in
this thread.

It looks well.

Thank you

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

#119Erik Rijkers
er@xs4all.nl
In reply to: Daniel Gustafsson (#117)
Re: proposal: possibility to read dumped table's name from file

On 10/1/21 3:19 PM, Daniel Gustafsson wrote:

As has been discussed upthread, this format strikes a compromise wrt simplicity
and doesn't preclude adding a more structured config file in the future should
we want that. I think this takes care of most comments and opinions made in
this thread.

--
Daniel Gustafsson https://vmware.com/

Hi,

If you try to dump/restore a foreign file from a file_fdw server, the
restore step will complain and thus leave the returnvalue nonzero. The
foreign table will be there, with complete 'data'.

A complete runnable exampe is a lot of work; I hope the below bits of
input and output makes the problem clear. Main thing: the pg_restore
contains 2 ERROR lines like:

pg_restore: error: COPY failed for table "ireise1": ERROR: cannot
insert into foreign table "ireise1"

----------------------
From the test bash:

echo "
include table table0 # ok public
include table test.table1 #
include foreign_data goethe # foreign server 'goethe' (file_fdw)
include table gutenberg.ireise1 # foreign table
include table gutenberg.ireise2 # foreign table
" > inputfile1.txt

pg_dump --create -Fc -c -p $port -d $db1 -f dump1 --filter=inputfile1.txt
echo

# prepare for restore
server_name=goethe
echo "create schema if not exists test;" | psql -qaXd $db2
echo "create schema if not exists gutenberg;" | psql -qaXd $db2
echo "create server if not exists $server_name foreign data wrapper
file_fdw " \
| psql -qaXd $db2

echo "-- pg_restore --if-exists -cvd $db2 dump1 "
pg_restore --if-exists -cvd $db2 dump1
rc=$?
echo "-- rc [$rc]" -
echo

----------------------

from the output:

-- pg_dump --create -Fc -c -p 6969 -d testdb1 -f dump1
--filter=inputfile1.txt

-- pg_restore --if-exists -cvd testdb2 dump1
pg_restore: connecting to database for restore
pg_restore: dropping TABLE table1
pg_restore: dropping TABLE table0
pg_restore: dropping FOREIGN TABLE ireise2
pg_restore: dropping FOREIGN TABLE ireise1
pg_restore: creating FOREIGN TABLE "gutenberg.ireise1"
pg_restore: creating COMMENT "gutenberg.FOREIGN TABLE ireise1"
pg_restore: creating FOREIGN TABLE "gutenberg.ireise2"
pg_restore: creating COMMENT "gutenberg.FOREIGN TABLE ireise2"
pg_restore: creating TABLE "public.table0"
pg_restore: creating TABLE "test.table1"
pg_restore: processing data for table "gutenberg.ireise1"
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 5570; 0 23625 TABLE DATA ireise1 aardvark
pg_restore: error: COPY failed for table "ireise1": ERROR: cannot
insert into foreign table "ireise1"
pg_restore: processing data for table "gutenberg.ireise2"
pg_restore: from TOC entry 5571; 0 23628 TABLE DATA ireise2 aardvark
pg_restore: error: COPY failed for table "ireise2": ERROR: cannot
insert into foreign table "ireise2"
pg_restore: processing data for table "public.table0"
pg_restore: processing data for table "test.table1"
pg_restore: warning: errors ignored on restore: 2
-- rc [1]

---------

A second, separate practical hickup is that schema's are not restored
from the dumped $schema.$table includes -- but this can be worked
around; for my inputfile1.txt I had to run separately (as seen above,
before running the pg_restore):

create schema if not exists test;
create schema if not exists gutenberg;
create server if not exists goethe foreign data wrapper file_fdw;

A bit annoying but still maybe all right.

Thanks,

Erik Rijkers

#120Erik Rijkers
er@xs4all.nl
In reply to: Erik Rijkers (#119)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 10/1/21 6:19 PM, Erik Rijkers wrote:

On 10/1/21 3:19 PM, Daniel Gustafsson wrote:

As has been discussed upthread, this format strikes a compromise wrt
simplicity
and doesn't preclude adding a more structured config file in the
future should

If you try to dump/restore a foreign file from a file_fdw server, the
restore step will complain and thus leave the returnvalue nonzero. The
foreign table will be there, with complete 'data'.

A complete runnable exampe is a lot of work; I hope the below bits of
input and output makes the problem clear.� Main thing: the pg_restore
contains 2 ERROR lines like:

pg_restore: error: COPY failed for table "ireise1": ERROR:� cannot
insert into foreign table "ireise1"

Further testing makes clear that the file_fdw-addressing line
include foreign_data goethe
was the culprit: it causes a COPY which of course fails in a readonly
wrapper like file_fdw. Without that line it works (because I run the
restore on the same machine so the underlying file_fdw .txt files are
there for testdb2 too)

So the issue is not as serious as it seemed. The complaint remaining is
only that this could somehow be documented better.

I attach a running example (careful, it deletes stuff) of the original
ERROR-producing bash (remove the 'include foreign_data' line from the
input file to run it without error).

thanks,

Erik Rijkers

Attachments:

dump_restore_foreign_table.shapplication/x-shellscript; name=dump_restore_foreign_table.shDownload
#121Daniel Gustafsson
daniel@yesql.se
In reply to: Erik Rijkers (#120)
Re: proposal: possibility to read dumped table's name from file

On 2 Oct 2021, at 08:18, Erik Rijkers <er@xs4all.nl> wrote:

So the issue is not as serious as it seemed.

This is also not related to this patch in any way, or am I missing a point
here? This can just as well be achieved without this patch.

The complaint remaining is only that this could somehow be documented better.

The pg_dump documentation today have a large highlighted note about this:

"When --include-foreign-data is specified, pg_dump does not check that the
foreign table is writable. Therefore, there is no guarantee that the
results of a foreign table dump can be successfully restored."

This was extensively discussed [0]/messages/by-id/LEJPR01MB0185483C0079D2F651B16231E7FC0@LEJPR01MB0185.DEUPRD01.PROD.OUTLOOK.DE when this went in, is there additional
documentation you'd like to see for this?

--
Daniel Gustafsson https://vmware.com/

[0]: /messages/by-id/LEJPR01MB0185483C0079D2F651B16231E7FC0@LEJPR01MB0185.DEUPRD01.PROD.OUTLOOK.DE

#122Daniel Gustafsson
daniel@yesql.se
In reply to: Daniel Gustafsson (#117)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 1 Oct 2021, at 15:19, Daniel Gustafsson <daniel@yesql.se> wrote:

I'm still not happy with the docs, I need to take another look there and see if
I make them more readable but otherwise I don't think there are any open issues
with this.

Attached is a rebased version which has rewritten docs which I think are more
in line with the pg_dump documentation. I've also added tests for
--strict-names operation, as well subjected it to pgindent and pgperltidy.

Unless there are objections, I think this is pretty much ready to go in.

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20211005.patchapplication/octet-stream; name=pg_dump-filteropt-20211005.patch; x-unix-mode=0644Download
From 052f234e53626ca8906a972c89a1b55f67e22beb Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Fri, 1 Oct 2021 15:03:15 +0200
Subject: [PATCH] Add --filter option for reading object patterns from file

This adds the --filter=FILENAME option to pg_dump for specifying a set of
inclusion/exclusion patterns in a file.  In situations where the wanted
filterset exceeds the commandline capabilities, this can be a handy way
to still be able to filter objects. The format of the file is line based
with each pattern on its own line:

<command> <object_type> <pattern>

When object names contain whitespace the pattern must be quoted.

The object_types table, schema, foreign_data and data each correspond
to the existing command line parameters -t/--table, -n/--schema,
--include-foreign-data and --exclude-table-data.

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com

fixup
---
 doc/src/sgml/ref/pg_dump.sgml               |  88 ++++
 src/bin/pg_dump/pg_dump.c                   | 419 ++++++++++++++++++++
 src/bin/pg_dump/t/004_pg_dump_filterfile.pl | 399 +++++++++++++++++++
 3 files changed, 906 insertions(+)
 create mode 100644 src/bin/pg_dump/t/004_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..1c60f24b1d 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>.  This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>.  This keyword can only be
+           with the <literal>exclude</literal>  keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1122,6 +1196,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1531,6 +1606,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a485fb2d07..796c431aaa 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -90,6 +92,28 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+}			FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -308,6 +332,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -380,6 +405,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +639,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1068,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18979,3 +19011,390 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * exit_invalid_filter_format - Emit error message, close the file and exit
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		if (fclose(fstate->fp) != 0)
+			fatal("could not close filter file \"%s\": %m", fstate->filename);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic characters) in
+ * the passed in line buffer.  Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern.  Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	StringInfoData line;
+
+	/* We only allocate the buffer if we need it */
+	line.data = NULL;
+
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name pattern");
+
+	/*
+	 * If the object name pattern has been quoted we must take care parse out
+	 * the entire quoted pattern, which may contain whitespace and can span
+	 * over many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				if (line.data == NULL)
+					initStringInfo(&line);
+
+				if (!pg_get_line_buf(fstate->fp, &line))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						if (fstate->fp != stdin)
+						{
+							if (fclose(fstate->fp) != 0)
+								fatal("could not close filter file \"%s\": %m",
+									  fstate->filename);
+						}
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate, "unexpected end of file");
+				}
+
+				str = line.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	if (line.data != NULL)
+		pg_free(line.data);
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a row
+ * based format a pattern may span more than one line due to how object names
+ * can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of: "table",
+ * "schema", "foreign_data" or "data". The pattern is either simple without any
+ * whitespace, or properly quoted in case there is whitespace in the object
+ * name. The pattern handling follows the same rules as other object include
+ * and exclude functions; it can use wildcards. Returns true, when one filter
+ * item was successfully read and parsed.  When object name contains \n chars,
+ * then more than one line from input file can be processed. Returns false when
+ * the filter file reaches EOF.  In case of errors, the function wont return
+ * but will exit with an appropriate error message.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	StringInfoData line;
+
+	initStringInfo(&line);
+
+	if (pg_get_line_buf(fstate->fp, &line))
+	{
+		char	   *str = line.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no filtercommand found (expected \"include\" or \"exclude\")");
+
+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid filtercommand (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no object type found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid object type (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_pattern(fstate, str, objname);
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+										   "unexpected extra data after pattern");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		free(line.data);
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+		{
+			if (fclose(fstate->fp) != 0)
+				fatal("could not close filter file \"%s\": %m", fstate->filename);
+		}
+
+		exit_nicely(1);
+	}
+
+	free(line.data);
+
+	return false;
+}
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+										   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..28ae6f1db8
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,399 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 49;
+
+my $tempdir = TestLib::tempdir;
+my $inputfile;
+
+my $node      = PostgresNode->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filtercommand/,
+	"invalid syntax: incorrect filtercommand");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid object type/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
-- 
2.24.3 (Apple Git-128)

#123Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#122)
Re: proposal: possibility to read dumped table's name from file

út 5. 10. 2021 v 14:30 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 1 Oct 2021, at 15:19, Daniel Gustafsson <daniel@yesql.se> wrote:

I'm still not happy with the docs, I need to take another look there and

see if

I make them more readable but otherwise I don't think there are any open

issues

with this.

Attached is a rebased version which has rewritten docs which I think are
more
in line with the pg_dump documentation. I've also added tests for
--strict-names operation, as well subjected it to pgindent and pgperltidy.

Unless there are objections, I think this is pretty much ready to go in.

great, thank you

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

#124Erik Rijkers
er@xs4all.nl
In reply to: Daniel Gustafsson (#122)
Re: proposal: possibility to read dumped table's name from file

Op 05-10-2021 om 14:30 schreef Daniel Gustafsson:

Unless there are objections, I think this is pretty much ready to go in.

Agreed. One typo:

'This keyword can only be with the exclude keyword.' should be
'This keyword can only be used with the exclude keyword.'

thanks,

Erik Rijkers

Show quoted text

--
Daniel Gustafsson https://vmware.com/

#125Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#122)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

út 5. 10. 2021 v 14:30 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 1 Oct 2021, at 15:19, Daniel Gustafsson <daniel@yesql.se> wrote:

I'm still not happy with the docs, I need to take another look there and

see if

I make them more readable but otherwise I don't think there are any open

issues

with this.

Attached is a rebased version which has rewritten docs which I think are
more
in line with the pg_dump documentation. I've also added tests for
--strict-names operation, as well subjected it to pgindent and pgperltidy.

Unless there are objections, I think this is pretty much ready to go in.

I am sending a rebased version of patch pg_dump-filteropt-20211005.patch
with fixed regress tests and fixed documentation (reported by Erik).
I found another issue - the stringinfo line used in filter_get_pattern was
released too early - the line (memory) was used later in check of unexpected
chars after pattern string. I fixed it by moving this stringinfo buffer to
fstate structure. It can be shared by all routines, and it can be safely
released at
an end of filter processing, where we are sure, so these data can be free.

Regards

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

Attachments:

pg_dump-filteropt-20211027.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20211027.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 7682226b99..9245355686 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -789,6 +789,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1122,6 +1196,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1531,6 +1606,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d1842edde0..f4f84bdce0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -90,6 +92,29 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+}			FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -308,6 +333,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -380,6 +406,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -613,6 +640,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
 				exit_nicely(1);
@@ -1038,6 +1069,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18822,3 +18855,376 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse reloptions array");
 }
+
+/*
+ * exit_invalid_filter_format - Emit error message, close the file and exit
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		if (fclose(fstate->fp) != 0)
+			fatal("could not close filter file \"%s\": %m", fstate->filename);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic characters) in
+ * the passed in line buffer.  Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern.  Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name pattern");
+
+	/*
+	 * If the object name pattern has been quoted we must take care parse out
+	 * the entire quoted pattern, which may contain whitespace and can span
+	 * over many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						if (fstate->fp != stdin)
+						{
+							if (fclose(fstate->fp) != 0)
+								fatal("could not close filter file \"%s\": %m",
+									  fstate->filename);
+						}
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate, "unexpected end of file");
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a row
+ * based format a pattern may span more than one line due to how object names
+ * can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of: "table",
+ * "schema", "foreign_data" or "data". The pattern is either simple without any
+ * whitespace, or properly quoted in case there is whitespace in the object
+ * name. The pattern handling follows the same rules as other object include
+ * and exclude functions; it can use wildcards. Returns true, when one filter
+ * item was successfully read and parsed.  When object name contains \n chars,
+ * then more than one line from input file can be processed. Returns false when
+ * the filter file reaches EOF.  In case of errors, the function wont return
+ * but will exit with an appropriate error message.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no filtercommand found (expected \"include\" or \"exclude\")");
+
+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid filtercommand (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no object type found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid object type (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_pattern(fstate, str, objname);
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+										   "unexpected extra data after pattern");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+		{
+			if (fclose(fstate->fp) != 0)
+				fatal("could not close filter file \"%s\": %m", fstate->filename);
+		}
+
+		exit_nicely(1);
+	}
+
+	return false;
+}
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+	initStringInfo(&fstate.linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+										   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	free(fstate.linebuff.data);
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..44bc3e8e96
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,398 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 49;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filtercommand/,
+	"invalid syntax: incorrect filtercommand");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid object type/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
#126Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#97)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

fresh rebase

Regards

Pavel

Attachments:

pg_dump-filteropt-20220425.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt-20220425.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c946755737..3711959fa2 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 786d592e2b..bb00e4fd4b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -55,10 +55,12 @@
 #include "catalog/pg_trigger_d.h"
 #include "catalog/pg_type_d.h"
 #include "common/connect.h"
+#include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
 #include "getopt_long.h"
+#include "lib/stringinfo.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
 #include "pg_backup_db.h"
@@ -96,6 +98,29 @@ typedef enum OidOptions
 	zeroAsNone = 4
 } OidOptions;
 
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_DATA
+}			FilterObjectType;
+
 /* global decls */
 static bool dosync = true;		/* Issue fsync() to make dump durable on disk. */
 
@@ -317,6 +342,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -389,6 +415,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +649,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1058,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18132,3 +18165,376 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * exit_invalid_filter_format - Emit error message, close the file and exit
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		if (fclose(fstate->fp) != 0)
+			pg_fatal("could not close filter file \"%s\": %m", fstate->filename);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic characters) in
+ * the passed in line buffer.  Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern.  Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name pattern");
+
+	/*
+	 * If the object name pattern has been quoted we must take care parse out
+	 * the entire quoted pattern, which may contain whitespace and can span
+	 * over many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						if (fstate->fp != stdin)
+						{
+							if (fclose(fstate->fp) != 0)
+								pg_fatal("could not close filter file \"%s\": %m",
+									  fstate->filename);
+						}
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate, "unexpected end of file");
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a row
+ * based format a pattern may span more than one line due to how object names
+ * can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of: "table",
+ * "schema", "foreign_data" or "data". The pattern is either simple without any
+ * whitespace, or properly quoted in case there is whitespace in the object
+ * name. The pattern handling follows the same rules as other object include
+ * and exclude functions; it can use wildcards. Returns true, when one filter
+ * item was successfully read and parsed.  When object name contains \n chars,
+ * then more than one line from input file can be processed. Returns false when
+ * the filter file reaches EOF.  In case of errors, the function wont return
+ * but will exit with an appropriate error message.
+ */
+static bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no filtercommand found (expected \"include\" or \"exclude\")");
+
+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid filtercommand (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no object type found (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid object type (expected \"table\", \"schema\", \"foreign_data\" or \"data\")");
+
+			str = filter_get_pattern(fstate, str, objname);
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+										   "unexpected extra data after pattern");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+		{
+			if (fclose(fstate->fp) != 0)
+				pg_fatal("could not close filter file \"%s\": %m", fstate->filename);
+		}
+
+		exit_nicely(1);
+	}
+
+	return false;
+}
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+	initStringInfo(&fstate.linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			pg_fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+										   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	free(fstate.linebuff.data);
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			pg_fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..44bc3e8e96
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,398 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 49;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filtercommand/,
+	"invalid syntax: incorrect filtercommand");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid object type/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
#127Andrew Dunstan
andrew@dunslane.net
In reply to: Pavel Stehule (#126)
Re: proposal: possibility to read dumped table's name from file

On 2022-04-25 Mo 13:39, Pavel Stehule wrote:

Hi

fresh rebase

If we're going to do this for pg_dump's include/exclude options,
shouldn't we also provide an equivalent facility for pg_dumpall's
--exclude-database option?

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

#128Pavel Stehule
pavel.stehule@gmail.com
In reply to: Andrew Dunstan (#127)
Re: proposal: possibility to read dumped table's name from file

st 13. 7. 2022 v 22:49 odesílatel Andrew Dunstan <andrew@dunslane.net>
napsal:

On 2022-04-25 Mo 13:39, Pavel Stehule wrote:

Hi

fresh rebase

If we're going to do this for pg_dump's include/exclude options,
shouldn't we also provide an equivalent facility for pg_dumpall's
--exclude-database option?

It has sense

Regards

Pavel

Show quoted text

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

#129Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#128)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

čt 14. 7. 2022 v 6:54 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

st 13. 7. 2022 v 22:49 odesílatel Andrew Dunstan <andrew@dunslane.net>
napsal:

On 2022-04-25 Mo 13:39, Pavel Stehule wrote:

Hi

fresh rebase

If we're going to do this for pg_dump's include/exclude options,
shouldn't we also provide an equivalent facility for pg_dumpall's
--exclude-database option?

It has sense

The attached patch implements the --filter option for pg_dumpall and for
pg_restore too.

Regards

Pavel

Show quoted text

Attachments:

pg_dump-filteropt.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 5efb442b44..ba2920dbee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 8a081f0080..137491340c 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,29 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        like <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 526986eadb..5f16c4a333 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules like <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2f524b09bf..373c4caa86 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -25,6 +25,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -42,8 +43,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o pg_backup_utils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o pg_backup_utils.o  $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..b09caef1f3
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,369 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"
+#include "common/fe_memutils.h"
+#include "common/string.h"
+#include "lib/stringinfo.h"
+#include "pg_backup_utils.h"
+#include "pqexpbuffer.h"
+
+/*
+ * exit_invalid_filter_format - Emit error message, close the file and exit
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+exit_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+
+		if (fclose(fstate->fp) != 0)
+			pg_fatal("could not close filter file \"%s\": %m", fstate->filename);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	exit_nicely(1);
+}
+
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_DATA:
+			return "data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	return "unknown object type";
+}
+
+/*
+ * Helper routine to reduce duplicated code
+ */
+void
+exit_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "The application \"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	exit_invalid_filter_format(fstate, str->data);
+}
+
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic characters) in
+ * the passed in line buffer.  Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern.  Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+		exit_invalid_filter_format(fstate, "missing object name pattern");
+
+	/*
+	 * If the object name pattern has been quoted we must take care parse out
+	 * the entire quoted pattern, which may contain whitespace and can span
+	 * over many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						if (fstate->fp != stdin)
+						{
+							if (fclose(fstate->fp) != 0)
+								pg_fatal("could not close filter file \"%s\": %m",
+									  fstate->filename);
+						}
+
+						exit_nicely(1);
+					}
+
+					exit_invalid_filter_format(fstate, "unexpected end of file");
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a row
+ * based format a pattern may span more than one line due to how object names
+ * can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of: "table",
+ * "schema", "foreign_data", "data", "database", "function", "trigger" or "index".
+ * The pattern is either simple without any  whitespace, or properly quoted
+ * in case there is whitespace in the object name. The pattern handling follows
+ * the same rules as other object include and exclude functions; it can use
+ * wildcards. Returns true, when one filter item was successfully read and
+ * parsed.  When object name contains \n chars, then more than one line from
+ * input file can be processed. Returns false when the filter file reaches EOF.
+ * In case of errors, the function wont return but will exit with an appropriate
+ * error message.
+ */
+bool
+read_filter_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+
+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;
+			else
+				exit_invalid_filter_format(fstate,
+										   "invalid filter command (expected \"include\" or \"exclude\")");
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+				exit_invalid_filter_format(fstate, "missing filter object type");
+
+			if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else if (size == 8 && pg_strncasecmp(keyword, "database", 8) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 8 && pg_strncasecmp(keyword, "function", 8) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (size == 5 && pg_strncasecmp(keyword, "index", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 7 && pg_strncasecmp(keyword, "trigger", 7) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				exit_invalid_filter_format(fstate, str->data);
+			}
+
+			str = filter_get_pattern(fstate, str, objname);
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+				exit_invalid_filter_format(fstate,
+										   "unexpected extra data after pattern");
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+
+		if (fstate->fp != stdin)
+		{
+			if (fclose(fstate->fp) != 0)
+				pg_fatal("could not close filter file \"%s\": %m", fstate->filename);
+		}
+
+		exit_nicely(1);
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..e4a1a74b10
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,51 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern void exit_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void exit_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool read_filter_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
\ No newline at end of file
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e4fdb6b75b..cbe712226a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -317,6 +318,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -389,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18130,3 +18139,93 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+	initStringInfo(&fstate.linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			pg_fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+				exit_invalid_filter_format(&fstate,
+										   "include filter is not allowed for this type of object");
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+			exit_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+
+		if (objname)
+			free(objname);
+	}
+
+	free(fstate.linebuff.data);
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			pg_fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 26d3d53809..1f75e40b49 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -25,6 +25,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -56,6 +57,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -133,6 +135,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -335,6 +338,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -628,6 +636,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1748,3 +1757,60 @@ dumpTimestamp(const char *msg)
 	if (strftime(buf, sizeof(buf), PGDUMP_STRFTIME_FMT, localtime(&now)) != 0)
 		fprintf(OPF, "-- %s %s\n\n", msg, buf);
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+	initStringInfo(&fstate.linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			pg_fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+				exit_invalid_filter_format(&fstate,
+										   "include database filter is not suppported");
+		}
+		else
+			exit_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+
+		if (objname)
+			free(objname);
+	}
+
+	free(fstate.linebuff.data);
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			pg_fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..31634c2396 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void getFiltersFromFile(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				getFiltersFromFile(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,108 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	fstate.filename = filename;
+	fstate.lineno = 0;
+	initStringInfo(&fstate.linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate.fp = fopen(filename, "r");
+		if (!fstate.fp)
+			pg_fatal("could not open filter file \"%s\": %m", filename);
+	}
+	else
+		fstate.fp = stdin;
+
+	while (read_filter_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, optarg);
+			}
+			else
+				exit_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+		}
+		else
+			exit_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+
+		if (objname)
+			free(objname);
+	}
+
+	free(fstate.linebuff.data);
+
+	if (fstate.fp != stdin)
+	{
+		if (fclose(fstate.fp) != 0)
+			pg_fatal("could not close filter file \"%s\": %m", fstate.filename);
+	}
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..246880f6ec
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,398 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 49;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
#130Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#129)
Re: proposal: possibility to read dumped table's name from file

Thanks for updating the patch.

This failed to build on windows.
http://cfbot.cputube.org/pavel-stehule.html

Some more comments inline.

On Sun, Jul 17, 2022 at 08:20:47AM +0200, Pavel Stehule wrote:

The attached patch implements the --filter option for pg_dumpall and for
pg_restore too.

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 5efb442b44..ba2920dbee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the

STDIN comma

+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        are ignored. Comments can be placed after filter as well. Blank lines

change "are ignored" to "ignored", I think.

diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 8a081f0080..137491340c 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,29 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        like <option>--exclude-database</option>.

same rules *as*

+ To read from <literal>STDIN</literal> use <filename>-</filename> as the

comma

+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding

For dumpall, remove "for including or"
change "above listed options" to "exclude-database" ?

diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 526986eadb..5f16c4a333 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules like <option>--schema</option>, <option>--exclude-schema</option>,

s/like/as/

+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal> use <filename>-</filename> as the

STDIN comma

+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic characters) in

remove "containing"

+	/*
+	 * If the object name pattern has been quoted we must take care parse out
+	 * the entire quoted pattern, which may contain whitespace and can span
+	 * over many lines.

quoted comma
*to parse
remove "over"

+ * The pattern is either simple without any whitespace, or properly quoted

double space

+ * in case there is whitespace in the object name. The pattern handling follows

s/is/may be/

+			if (size == 7 && pg_strncasecmp(keyword, "include", 7) == 0)
+				*is_include = true;
+			else if (size == 7 && pg_strncasecmp(keyword, "exclude", 7) == 0)
+				*is_include = false;

Can't you write strncasecmp(keyword, "include", size) to avoid hardcoding "7" ?

+
+			if (size == 4 && pg_strncasecmp(keyword, "data", 4) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else if (size == 8 && pg_strncasecmp(keyword, "database", 8) == 0)
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (size == 12 && pg_strncasecmp(keyword, "foreign_data", 12) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (size == 8 && pg_strncasecmp(keyword, "function", 8) == 0)
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (size == 5 && pg_strncasecmp(keyword, "index", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (size == 6 && pg_strncasecmp(keyword, "schema", 6) == 0)
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (size == 5 && pg_strncasecmp(keyword, "table", 5) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (size == 7 && pg_strncasecmp(keyword, "trigger", 7) == 0)
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;

Avoid hardcoding these constants.

diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..e4a1a74b10
--- /dev/null
+++ b/src/bin/pg_dump/filter.h

...

\ No newline at end of file

:(

#131Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#130)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

ne 17. 7. 2022 v 16:01 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

Thanks for updating the patch.

This failed to build on windows.
http://cfbot.cputube.org/pavel-stehule.html

Yes, there was a significant problem with the function exit_nicely, that is
differently implemented in pg_dump and pg_dumpall.

Some more comments inline.

On Sun, Jul 17, 2022 at 08:20:47AM +0200, Pavel Stehule wrote:

The attached patch implements the --filter option for pg_dumpall and for
pg_restore too.

diff --git a/doc/src/sgml/ref/pg_dump.sgml

b/doc/src/sgml/ref/pg_dump.sgml

index 5efb442b44..ba2920dbee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable

class="parameter">filename</replaceable></option></term>

+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to

include

+ or exclude from the dump. The patterns are interpreted

according to the

+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign

servers and

+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal> use

<filename>-</filename> as the

STDIN comma

fixed

+       <para>
+        Lines starting with <literal>#</literal> are considered

comments and

+ are ignored. Comments can be placed after filter as well. Blank

lines

change "are ignored" to "ignored", I think.

changed

diff --git a/doc/src/sgml/ref/pg_dumpall.sgml

b/doc/src/sgml/ref/pg_dumpall.sgml

index 8a081f0080..137491340c 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,29 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable

class="parameter">filename</replaceable></option></term>

+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases

excluded

+ from dump. The patterns are interpretted according to the same

rules

+ like <option>--exclude-database</option>.

same rules *as*

fixed

+ To read from <literal>STDIN</literal> use

<filename>-</filename> as the

comma

fixed

+ filename. The <option>--filter</option> option can be

specified in

+ conjunction with the above listed options for including or

excluding

For dumpall, remove "for including or"
change "above listed options" to "exclude-database" ?

fixed

diff --git a/doc/src/sgml/ref/pg_restore.sgml

b/doc/src/sgml/ref/pg_restore.sgml

index 526986eadb..5f16c4a333 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+     <varlistentry>
+      <term><option>--filter=<replaceable

class="parameter">filename</replaceable></option></term>

+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects

excluded

+ or included from restore. The patterns are interpretted

according to the

+ same rules like <option>--schema</option>,

<option>--exclude-schema</option>,

s/like/as/

changed

+ <option>--function</option>, <option>--index</option>,

<option>--table</option>

+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal> use

<filename>-</filename> as the

STDIN comma

fixed

+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to containing ascii alphabetic

characters) in

remove "containing"

fixed

+     /*
+      * If the object name pattern has been quoted we must take care

parse out

+ * the entire quoted pattern, which may contain whitespace and can

span

+ * over many lines.

quoted comma
*to parse
remove "over"

fixed

+ * The pattern is either simple without any whitespace, or properly

quoted

double space

fixed

+ * in case there is whitespace in the object name. The pattern handling

follows

s/is/may be/

fixed

+ if (size == 7 && pg_strncasecmp(keyword,

"include", 7) == 0)

+                             *is_include = true;
+                     else if (size == 7 && pg_strncasecmp(keyword,

"exclude", 7) == 0)

+ *is_include = false;

Can't you write strncasecmp(keyword, "include", size) to avoid hardcoding
"7" ?

I need to compare the size of the keyword with expected size, but I can use
strlen(conststr). I wrote new macro is_keyword_str to fix this issue

fixed

+
+                     if (size == 4 && pg_strncasecmp(keyword, "data",

4) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_DATA;
+                     else if (size == 8 && pg_strncasecmp(keyword,

"database", 8) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_DATABASE;
+                     else if (size == 12 && pg_strncasecmp(keyword,

"foreign_data", 12) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+                     else if (size == 8 && pg_strncasecmp(keyword,

"function", 8) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_FUNCTION;
+                     else if (size == 5 && pg_strncasecmp(keyword,

"index", 5) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_INDEX;
+                     else if (size == 6 && pg_strncasecmp(keyword,

"schema", 6) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_SCHEMA;
+                     else if (size == 5 && pg_strncasecmp(keyword,

"table", 5) == 0)

+                             *objtype = FILTER_OBJECT_TYPE_TABLE;
+                     else if (size == 7 && pg_strncasecmp(keyword,

"trigger", 7) == 0)

+ *objtype = FILTER_OBJECT_TYPE_TRIGGER;

Avoid hardcoding these constants.

fixed

diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..e4a1a74b10
--- /dev/null
+++ b/src/bin/pg_dump/filter.h

...

\ No newline at end of file

fixed

updated patch attached

Regards

Pavel

Show quoted text

:(

Attachments:

pg_dump-filteropt.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filteropt.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 5efb442b44..5bcbcf17eb 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 8a081f0080..799910a487 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 526986eadb..d3ca0b36d5 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2f524b09bf..d8c1babc26 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -25,6 +25,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -42,8 +43,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..5a76fd57f1
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,431 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"
+#include "common/fe_memutils.h"
+#include "common/string.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == bytes) && (pg_strncasecmp(cstr, str, bytes) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this rutine in pg_dumpall. So instead
+ * direct calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.
+ */
+
+/*
+ * Simple routines - just don't repeat same code
+ *
+ * Returns true, when filter's file is opened
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated sources for filter
+ */
+void
+filter_free_sources(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * log_format_error - Emit error message
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_DATA:
+			return "data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	return "unknown object type";
+}
+
+/*
+ * Helper routine to reduce duplicated code
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "The application \"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern. Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	/*
+	 * If the object name pattern has been quoted, we must take care to parse
+	 * out the entire quoted pattern, which may contain whitespace and can span
+	 * many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						fstate->is_error = true;
+					}
+					else
+						log_invalid_filter_format(fstate, "unexpected end of file");
+
+					return NULL;
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of:
+ * "table", "schema", "foreign_data", "data", "database", "function",
+ * "trigger" or "index". The pattern is either simple without any
+ * whitespace, or properly quoted in case there may be whitespace in the
+ * object name. The pattern handling follows the same rules as other object
+ * include and exclude functions; it can use wildcards. Returns true, when
+ * one filter item was successfully read and parsed.  When object name
+ * contains \n chars, then more than one line from input file can be
+ * processed. Returns false when the filter file reaches EOF. In case of
+ * errors, the function wont return but will exit with an appropriate error
+ * message.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			str = filter_get_pattern(fstate, str, objname);
+			if (!str)
+				return false;
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+			{
+				log_invalid_filter_format(fstate,
+										  "unexpected extra data after pattern");
+				return false;
+			}
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..f2503286aa
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free_sources(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e4fdb6b75b..040f5b07ed 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -317,6 +318,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -389,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18130,3 +18139,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "include filter is not allowed for this type of object");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 26d3d53809..d87d29093f 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -25,6 +25,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -56,6 +57,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -133,6 +135,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -335,6 +338,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -628,6 +636,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1735,7 +1744,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1748,3 +1756,53 @@ dumpTimestamp(const char *msg)
 	if (strftime(buf, sizeof(buf), PGDUMP_STRFTIME_FMT, localtime(&now)) != 0)
 		fprintf(OPF, "-- %s %s\n\n", msg, buf);
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "include database filter is not suppported");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..ab8525cafd 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void getFiltersFromFile(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				getFiltersFromFile(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, optarg);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..246880f6ec
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,398 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 49;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index e4feda10fd..ccd36d5634 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -446,6 +446,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#132Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#131)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

I am sending fresh rebase + enhancing tests for pg_dumpall and pg_restore

Regards

Pavel

Attachments:

pg_dump_filteropt.patchtext/x-patch; charset=US-ASCII; name=pg_dump_filteropt.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c08276bc0a..b64bae6987 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5d54074e01..7671795060 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2f524b09bf..d8c1babc26 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -25,6 +25,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -42,8 +43,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..5a76fd57f1
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,431 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"
+#include "common/fe_memutils.h"
+#include "common/string.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == bytes) && (pg_strncasecmp(cstr, str, bytes) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this rutine in pg_dumpall. So instead
+ * direct calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.
+ */
+
+/*
+ * Simple routines - just don't repeat same code
+ *
+ * Returns true, when filter's file is opened
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated sources for filter
+ */
+void
+filter_free_sources(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * log_format_error - Emit error message
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_DATA:
+			return "data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	return "unknown object type";
+}
+
+/*
+ * Helper routine to reduce duplicated code
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "The application \"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern. Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	/*
+	 * If the object name pattern has been quoted, we must take care to parse
+	 * out the entire quoted pattern, which may contain whitespace and can span
+	 * many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						fstate->is_error = true;
+					}
+					else
+						log_invalid_filter_format(fstate, "unexpected end of file");
+
+					return NULL;
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of:
+ * "table", "schema", "foreign_data", "data", "database", "function",
+ * "trigger" or "index". The pattern is either simple without any
+ * whitespace, or properly quoted in case there may be whitespace in the
+ * object name. The pattern handling follows the same rules as other object
+ * include and exclude functions; it can use wildcards. Returns true, when
+ * one filter item was successfully read and parsed.  When object name
+ * contains \n chars, then more than one line from input file can be
+ * processed. Returns false when the filter file reaches EOF. In case of
+ * errors, the function wont return but will exit with an appropriate error
+ * message.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			str = filter_get_pattern(fstate, str, objname);
+			if (!str)
+				return false;
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+			{
+				log_invalid_filter_format(fstate,
+										  "unexpected extra data after pattern");
+				return false;
+			}
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..f2503286aa
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free_sources(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2c68915732..533e078889 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -317,6 +318,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -389,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18164,3 +18173,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "include filter is not allowed for this type of object");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 26d3d53809..d87d29093f 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -25,6 +25,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -56,6 +57,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -133,6 +135,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -335,6 +338,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -628,6 +636,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1735,7 +1744,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1748,3 +1756,53 @@ dumpTimestamp(const char *msg)
 	if (strftime(buf, sizeof(buf), PGDUMP_STRFTIME_FMT, localtime(&now)) != 0)
 		fprintf(OPF, "-- %s %s\n\n", msg, buf);
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "include database filter is not suppported");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..ab8525cafd 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void getFiltersFromFile(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				getFiltersFromFile(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, optarg);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..dd533d79eb
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,491 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 62;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/The application "pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted tables are not restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index ee963d85f3..a24de02e31 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -454,6 +454,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#133Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#132)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

As noted upthread at some point, I'm not overly excited about the parser in
filter.c, for maintainability and readability reasons. So, I've reimplemented
the parser in Flex/Bison in the attached patch, which IMHO provides a clear(er)
picture of the grammar and is more per project standards. This version of the
patch is your latest version with just the parser replaced (at a reduction in
size as a side benefit).

All features supported in your latest patch version are present, and it passes
all the tests added by this patch. It's been an undisclosed amount of years
since I wrote a Bison parser (well, yacc really) from scratch so I don't rule
out having made silly mistakes. I would very much appreciate review from those
more well versed in this area.

One thing this patchversion currently lacks is refined error messaging, but if
we feel that this approach is a viable path then that can be tweaked. The
function which starts the parser can also be refactored to be shared across
pg_dump, pg_dumpall and pg_restore but I've kept it simple for now.

Thoughts? It would be nice to get this patch across the finishline during this
commitfest.

--
Daniel Gustafsson https://vmware.com/

Attachments:

0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchapplication/octet-stream; name=0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch; x-unix-mode=0644Download
From b683523b1477936c1381994e3a69bb28c512a3c1 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 doc/src/sgml/ref/pg_dump.sgml    |  88 ++++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml |  25 ++++++
 src/bin/pg_dump/.gitignore       |   4 +
 src/bin/pg_dump/Makefile         |  17 +++-
 src/bin/pg_dump/filter.h         |  44 ++++++++++
 src/bin/pg_dump/filterparse.y    |  65 +++++++++++++++
 src/bin/pg_dump/filterscan.l     | 133 +++++++++++++++++++++++++++++++
 src/bin/pg_dump/pg_dump.c        |  89 +++++++++++++++++++++
 src/bin/pg_dump/pg_dumpall.c     |  65 ++++++++++++++-
 src/bin/pg_dump/pg_restore.c     |  91 +++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm      |   4 +-
 12 files changed, 643 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c08276bc0a..b64bae6987 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5d54074e01..46a667c17a 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d7812779..11f2d68bea 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..e3befdc9b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..5dff4161f0
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 0000000000..16623ff824
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,65 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Patterns
+
+%%
+
+Patterns:
+	/* EMPTY */
+	| Patterns Pattern
+	| Pattern
+	;
+
+Pattern:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 0000000000..472645ac35
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,133 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>data			{ BEGIN(type); return OBJ_DATA; }
+<command>database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>index			{ BEGIN(type); return OBJ_INDEX; }
+<command>schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>table			{ BEGIN(type); return OBJ_TABLE; }
+<command>trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* XXX: check for overflow */
+
+		/* Double the size of litbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 67b6d9079e..a20f4c0b94 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -317,6 +318,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -389,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18197,3 +18206,83 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\"", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) dopt);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 69ae027bd3..d3763603a4 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -79,6 +80,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -156,6 +158,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -358,6 +361,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -651,6 +659,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1888,7 +1897,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1912,3 +1920,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\"", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..d4e6b753e0 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void getFiltersFromFile(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				getFiltersFromFile(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,86 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, RestoreOptions *opts)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\"", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) opts);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 098bc3f1b0..6418db9261 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.32.1 (Apple Git-133)

#134Erik Rijkers
er@xs4all.nl
In reply to: Daniel Gustafsson (#133)
Re: proposal: possibility to read dumped table's name from file

Op 07-09-2022 om 21:45 schreef Daniel Gustafsson:

One thing this patchversion currently lacks is refined error messaging, but if
we feel that this approach is a viable path then that can be tweaked. The
function which starts the parser can also be refactored to be shared across
pg_dump, pg_dumpall and pg_restore but I've kept it simple for now.

Thoughts? It would be nice to get this patch across the finishline during this
commitfest.

[0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch]

This seems to dump & restore well (as Pavels patch does).

I did notice one peculiarity (in your patch) where for each table a few
spaces are omitted by pg_dump.

-------------
#! /bin/bash

psql -qXc "drop database if exists testdb2"
psql -qXc "create database testdb2"

echo "
create schema if not exists test;
create table table0 (id integer);
create table table1 (id integer);
insert into table0 select n from generate_series(1,2) as f(n);
insert into table1 select n from generate_series(1,2) as f(n);
" | psql -qXad testdb2

echo "include table table0" > inputfile1.txt

echo "include table table0
include table table1" > inputfile2.txt

# 1 table, emits 2 spaces
echo -ne ">"
pg_dump -F p -f plainfile1 --filter=inputfile1.txt -d testdb2
echo "<"

# 2 tables, emits 4 space
echo -ne ">"
pg_dump -F p -f plainfile2 --filter=inputfile2.txt -d testdb2
echo "<"

# dump without filter emits no spaces
echo -ne ">"
pg_dump -F c -f plainfile3 -t table0 -table1 -d testdb2
echo "<"
-------------

It's probably a small thing -- but I didn't find it.

thanks,

Erik Rijkers

Show quoted text

--
Daniel Gustafsson https://vmware.com/

#135Daniel Gustafsson
daniel@yesql.se
In reply to: Erik Rijkers (#134)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 8 Sep 2022, at 12:00, Erik Rijkers <er@xs4all.nl> wrote:

Op 07-09-2022 om 21:45 schreef Daniel Gustafsson:

One thing this patchversion currently lacks is refined error messaging, but if
we feel that this approach is a viable path then that can be tweaked. The
function which starts the parser can also be refactored to be shared across
pg_dump, pg_dumpall and pg_restore but I've kept it simple for now.
Thoughts? It would be nice to get this patch across the finishline during this
commitfest.

[0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch]

This seems to dump & restore well (as Pavels patch does).

Thanks for looking!

I did notice one peculiarity (in your patch) where for each table a few spaces are omitted by pg_dump.

Right, I had that on my TODO to fix before submitting but clearly forgot. It
boils down to consuming the space between commands and object types and object
patterns. The attached v2 fixes that.

--
Daniel Gustafsson https://vmware.com/

Attachments:

v2-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchapplication/octet-stream; name=v2-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch; x-unix-mode=0644Download
From 90e0ea85d2dd66969962be92c1c78cc587398896 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH v2] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 doc/src/sgml/ref/pg_dump.sgml    |  88 ++++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml |  25 ++++++
 src/bin/pg_dump/.gitignore       |   4 +
 src/bin/pg_dump/Makefile         |  17 +++-
 src/bin/pg_dump/filter.h         |  44 ++++++++++
 src/bin/pg_dump/filterparse.y    |  65 +++++++++++++++
 src/bin/pg_dump/filterscan.l     | 134 +++++++++++++++++++++++++++++++
 src/bin/pg_dump/pg_dump.c        |  89 ++++++++++++++++++++
 src/bin/pg_dump/pg_dumpall.c     |  65 ++++++++++++++-
 src/bin/pg_dump/pg_restore.c     |  91 +++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm      |   4 +-
 12 files changed, 644 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c08276bc0a..b64bae6987 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5d54074e01..46a667c17a 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d7812779..11f2d68bea 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..e3befdc9b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..5dff4161f0
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 0000000000..16623ff824
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,65 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Patterns
+
+%%
+
+Patterns:
+	/* EMPTY */
+	| Patterns Pattern
+	| Pattern
+	;
+
+Pattern:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 0000000000..d9ef8b21fc
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,134 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>({space}*)data			{ BEGIN(type); return OBJ_DATA; }
+<command>({space}*)database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>({space}*)foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>({space}*)function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>({space}*)index			{ BEGIN(type); return OBJ_INDEX; }
+<command>({space}*)schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>({space}*)table			{ BEGIN(type); return OBJ_TABLE; }
+<command>({space}*)trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{space}		{ /* ignore */ }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* XXX: check for overflow */
+
+		/* Double the size of litbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 67b6d9079e..a20f4c0b94 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -317,6 +318,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AHX);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void getFiltersFromFile(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -389,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				getFiltersFromFile(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18197,3 +18206,83 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, DumpOptions *dopt)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\"", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) dopt);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 69ae027bd3..d3763603a4 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -79,6 +80,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -156,6 +158,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -358,6 +361,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -651,6 +659,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1888,7 +1897,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1912,3 +1920,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\"", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..d4e6b753e0 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void getFiltersFromFile(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				getFiltersFromFile(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,86 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * getFiltersFromFile - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getFiltersFromFile(const char *filename, RestoreOptions *opts)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\"", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) opts);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 098bc3f1b0..6418db9261 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.32.1 (Apple Git-133)

#136Julien Rouhaud
rjuju123@gmail.com
In reply to: Daniel Gustafsson (#135)
Re: proposal: possibility to read dumped table's name from file

Hi,

On Thu, Sep 08, 2022 at 01:38:42PM +0200, Daniel Gustafsson wrote:

On 8 Sep 2022, at 12:00, Erik Rijkers <er@xs4all.nl> wrote:

Op 07-09-2022 om 21:45 schreef Daniel Gustafsson:

One thing this patchversion currently lacks is refined error messaging, but if
we feel that this approach is a viable path then that can be tweaked. The
function which starts the parser can also be refactored to be shared across
pg_dump, pg_dumpall and pg_restore but I've kept it simple for now.
Thoughts? It would be nice to get this patch across the finishline during this
commitfest.

[0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch]

This seems to dump & restore well (as Pavels patch does).

Thanks for looking!

I did notice one peculiarity (in your patch) where for each table a few spaces are omitted by pg_dump.

Right, I had that on my TODO to fix before submitting but clearly forgot. It
boils down to consuming the space between commands and object types and object
patterns. The attached v2 fixes that.

I only had a quick look at the parser, and one thing that strikes me is:

+Patterns:
+	/* EMPTY */
+	| Patterns Pattern
+	| Pattern
+	;
+
+Pattern:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }

It seems confusing to mix Pattern(s) (the rules) and pattern (the token).
Maybe instead using Include(s) or Item(s) on the bison side, and/or
name_pattern on the lexer side?

#137Daniel Gustafsson
daniel@yesql.se
In reply to: Julien Rouhaud (#136)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 8 Sep 2022, at 13:44, Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Sep 08, 2022 at 01:38:42PM +0200, Daniel Gustafsson wrote:

On 8 Sep 2022, at 12:00, Erik Rijkers <er@xs4all.nl> wrote:

I did notice one peculiarity (in your patch) where for each table a few spaces are omitted by pg_dump.

Right, I had that on my TODO to fix before submitting but clearly forgot. It
boils down to consuming the space between commands and object types and object
patterns. The attached v2 fixes that.

I only had a quick look at the parser,

Thanks for looking!

.. and one thing that strikes me is:

+Patterns:
+	/* EMPTY */
+	| Patterns Pattern
+	| Pattern
+	;
+
+Pattern:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }

It seems confusing to mix Pattern(s) (the rules) and pattern (the token).
Maybe instead using Include(s) or Item(s) on the bison side, and/or
name_pattern on the lexer side?

That makes a lot of sense, I renamed the rules in the parser but kept them in
the lexer since that seemed like the clearest scheme.

Also in the attached is a small refactoring to share parser init between
pg_dump and pg_restore (pg_dumpall shares little with these so not there for
now), buffer resize overflow calculation and some error message tweaking.

--
Daniel Gustafsson https://vmware.com/

Attachments:

v3-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchapplication/octet-stream; name=v3-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch; x-unix-mode=0644Download
From 6102f42501afc47b917afa583d88c2c22ead22d8 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH v3] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 doc/src/sgml/ref/pg_dump.sgml     |  88 +++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml  |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml  |  25 ++++++
 src/bin/pg_dump/.gitignore        |   4 +
 src/bin/pg_dump/Makefile          |  17 +++-
 src/bin/pg_dump/filter.h          |  44 ++++++++++
 src/bin/pg_dump/filterparse.y     |  65 ++++++++++++++
 src/bin/pg_dump/filterscan.l      | 139 ++++++++++++++++++++++++++++++
 src/bin/pg_dump/pg_backup_utils.c |  33 +++++++
 src/bin/pg_dump/pg_backup_utils.h |   1 +
 src/bin/pg_dump/pg_dump.c         |  56 ++++++++++++
 src/bin/pg_dump/pg_dumpall.c      |  65 +++++++++++++-
 src/bin/pg_dump/pg_restore.c      |  58 +++++++++++++
 src/tools/msvc/Mkvcbuild.pm       |   4 +-
 14 files changed, 617 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c08276bc0a..b64bae6987 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5d54074e01..46a667c17a 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d7812779..11f2d68bea 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..e3befdc9b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..5dff4161f0
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 0000000000..424719c012
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,65 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Filters
+
+%%
+
+Filters:
+	/* EMPTY */
+	| Filters Filter
+	| Filter
+	;
+
+Filter:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 0000000000..0d7341086b
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,139 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>({space}*)data			{ BEGIN(type); return OBJ_DATA; }
+<command>({space}*)database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>({space}*)foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>({space}*)function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>({space}*)index			{ BEGIN(type); return OBJ_INDEX; }
+<command>({space}*)schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>({space}*)table			{ BEGIN(type); return OBJ_TABLE; }
+<command>({space}*)trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{space}		{ /* ignore */ }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* Ensure that we don't overflow */
+		if (patternbufsize > (SIZE_MAX / 2))
+		{
+			fprintf(stderr, "out of memory\n");
+			exit(1);
+		}
+
+		/* Double the size of patternbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index e40890cb26..d6db156bae 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,6 +13,7 @@
  */
 #include "postgres_fe.h"
 
+#include "filter.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
@@ -102,3 +103,35 @@ exit_nicely(int code)
 
 	exit(code);
 }
+
+/*
+ * Retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+void
+parse_filters_from_file(const char *filename, void *priv)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse(priv);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 8173bb93cf..74b942e712 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -30,6 +30,7 @@ extern const char *progname;
 extern void set_dump_section(const char *arg, int *dumpSections);
 extern void on_exit_nicely(on_exit_nicely_callback function, void *arg);
 extern void exit_nicely(int code) pg_attribute_noreturn();
+extern void parse_filters_from_file(const char *filename, void *priv);
 
 /* In pg_dump, we modify pg_fatal to call exit_nicely instead of exit */
 #undef pg_fatal
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 67b6d9079e..d88adfe59a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -389,6 +390,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +624,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				parse_filters_from_file(optarg, (void *) &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1033,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18197,3 +18205,51 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 69ae027bd3..4e9d898387 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -79,6 +80,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -156,6 +158,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -358,6 +361,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -651,6 +659,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1888,7 +1897,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1912,3 +1920,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..70897d9056 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,6 +47,7 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
@@ -123,6 +124,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +288,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				parse_filters_from_file(optarg, (void *) opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +469,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +501,54 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 098bc3f1b0..6418db9261 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.32.1 (Apple Git-133)

#138Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#133)
Re: proposal: possibility to read dumped table's name from file

Hi

st 7. 9. 2022 v 21:46 odesílatel Daniel Gustafsson <daniel@yesql.se> napsal:

As noted upthread at some point, I'm not overly excited about the parser in
filter.c, for maintainability and readability reasons. So, I've
reimplemented
the parser in Flex/Bison in the attached patch, which IMHO provides a
clear(er)
picture of the grammar and is more per project standards. This version of
the
patch is your latest version with just the parser replaced (at a reduction
in
size as a side benefit).

All features supported in your latest patch version are present, and it
passes
all the tests added by this patch. It's been an undisclosed amount of
years
since I wrote a Bison parser (well, yacc really) from scratch so I don't
rule
out having made silly mistakes. I would very much appreciate review from
those
more well versed in this area.

One thing this patchversion currently lacks is refined error messaging,
but if
we feel that this approach is a viable path then that can be tweaked. The
function which starts the parser can also be refactored to be shared across
pg_dump, pg_dumpall and pg_restore but I've kept it simple for now.

Thoughts? It would be nice to get this patch across the finishline during
this
commitfest.

I have no objections to this, and thank you so you try to move this patch
forward.

Regards

Pavel

--

Show quoted text

Daniel Gustafsson https://vmware.com/

#139John Naylor
john.naylor@enterprisedb.com
In reply to: Daniel Gustafsson (#137)
Re: proposal: possibility to read dumped table's name from file

On Thu, Sep 8, 2022 at 7:32 PM Daniel Gustafsson <daniel@yesql.se> wrote:

[v3]

Note that the grammar has shift-reduce conflicts. If you run a fairly
recent Bison, you can show them like this:

bison -Wno-deprecated -Wcounterexamples -d -o filterparse.c filterparse.y

filterparse.y: warning: 2 shift/reduce conflicts [-Wconflicts-sr]
filterparse.y: warning: shift/reduce conflict on token C_INCLUDE
[-Wcounterexamples]
Example: • C_INCLUDE include_object pattern
Shift derivation
Filters
↳ 3: Filter
↳ 4: • C_INCLUDE include_object pattern
Reduce derivation
Filters
↳ 2: Filters Filter
↳ 1: ε • ↳ 4: C_INCLUDE include_object pattern
filterparse.y: warning: shift/reduce conflict on token C_EXCLUDE
[-Wcounterexamples]
Example: • C_EXCLUDE exclude_object pattern
Shift derivation
Filters
↳ 3: Filter
↳ 5: • C_EXCLUDE exclude_object pattern
Reduce derivation
Filters
↳ 2: Filters Filter
↳ 1: ε • ↳ 5: C_EXCLUDE exclude_object pattern

--
John Naylor
EDB: http://www.enterprisedb.com

#140Andrew Dunstan
andrew@dunslane.net
In reply to: John Naylor (#139)
Re: proposal: possibility to read dumped table's name from file

On Sep 9, 2022, at 5:53 PM, John Naylor <john.naylor@enterprisedb.com> wrote:

On Thu, Sep 8, 2022 at 7:32 PM Daniel Gustafsson <daniel@yesql.se> wrote:

[v3]

Note that the grammar has shift-reduce conflicts. If you run a fairly
recent Bison, you can show them like this:

bison -Wno-deprecated -Wcounterexamples -d -o filterparse.c filterparse.y

filterparse.y: warning: 2 shift/reduce conflicts [-Wconflicts-sr]
filterparse.y: warning: shift/reduce conflict on token C_INCLUDE
[-Wcounterexamples]
Example: • C_INCLUDE include_object pattern
Shift derivation
Filters
↳ 3: Filter
↳ 4: • C_INCLUDE include_object pattern
Reduce derivation
Filters
↳ 2: Filters Filter
↳ 1: ε • ↳ 4: C_INCLUDE include_object pattern
filterparse.y: warning: shift/reduce conflict on token C_EXCLUDE
[-Wcounterexamples]
Example: • C_EXCLUDE exclude_object pattern
Shift derivation
Filters
↳ 3: Filter
↳ 5: • C_EXCLUDE exclude_object pattern
Reduce derivation
Filters
↳ 2: Filters Filter
↳ 1: ε • ↳ 5: C_EXCLUDE exclude_object pattern

Looks like the last rule for Filters should not be there. I do wonder whether we should be using bison/flex here, seems like using a sledgehammer to crack a nut.

Cheers

Andrew

#141Daniel Gustafsson
daniel@yesql.se
In reply to: Andrew Dunstan (#140)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 9 Sep 2022, at 11:00, Andrew Dunstan <andrew@dunslane.net> wrote:

On Sep 9, 2022, at 5:53 PM, John Naylor <john.naylor@enterprisedb.com> wrote:

Note that the grammar has shift-reduce conflicts.

Looks like the last rule for Filters should not be there.

Correct, fixed in the attached.

I do wonder whether we should be using bison/flex here, seems like using a
sledgehammer to crack a nut.

I don't the capabilities of the tool is all that interesting compared to the
long term maintainability and readability of the source code. Personally I
think a simple Bison/Flex parser is easier to read and reason about than the
corresponding written in C.

--
Daniel Gustafsson https://vmware.com/

Attachments:

v4-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchapplication/octet-stream; name=v4-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch; x-unix-mode=0644Download
From 5d36ef7c3814e60268e3d0dba36856f207bd1704 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH v4] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 doc/src/sgml/ref/pg_dump.sgml     |  88 +++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml  |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml  |  25 ++++++
 src/bin/pg_dump/.gitignore        |   4 +
 src/bin/pg_dump/Makefile          |  17 +++-
 src/bin/pg_dump/filter.h          |  44 ++++++++++
 src/bin/pg_dump/filterparse.y     |  64 ++++++++++++++
 src/bin/pg_dump/filterscan.l      | 139 ++++++++++++++++++++++++++++++
 src/bin/pg_dump/pg_backup_utils.c |  33 +++++++
 src/bin/pg_dump/pg_backup_utils.h |   1 +
 src/bin/pg_dump/pg_dump.c         |  56 ++++++++++++
 src/bin/pg_dump/pg_dumpall.c      |  65 +++++++++++++-
 src/bin/pg_dump/pg_restore.c      |  58 +++++++++++++
 src/tools/msvc/Mkvcbuild.pm       |   4 +-
 14 files changed, 616 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c08276bc0a..b64bae6987 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5d54074e01..46a667c17a 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d7812779..11f2d68bea 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..e3befdc9b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..5dff4161f0
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 0000000000..c7f06f788d
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,64 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Filters
+
+%%
+
+Filters:
+	/* EMPTY */
+	| Filters Filter
+	;
+
+Filter:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 0000000000..0d7341086b
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,139 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>({space}*)data			{ BEGIN(type); return OBJ_DATA; }
+<command>({space}*)database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>({space}*)foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>({space}*)function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>({space}*)index			{ BEGIN(type); return OBJ_INDEX; }
+<command>({space}*)schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>({space}*)table			{ BEGIN(type); return OBJ_TABLE; }
+<command>({space}*)trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{space}		{ /* ignore */ }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* Ensure that we don't overflow */
+		if (patternbufsize > (SIZE_MAX / 2))
+		{
+			fprintf(stderr, "out of memory\n");
+			exit(1);
+		}
+
+		/* Double the size of patternbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index e40890cb26..d6db156bae 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,6 +13,7 @@
  */
 #include "postgres_fe.h"
 
+#include "filter.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
@@ -102,3 +103,35 @@ exit_nicely(int code)
 
 	exit(code);
 }
+
+/*
+ * Retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+void
+parse_filters_from_file(const char *filename, void *priv)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse(priv);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 8173bb93cf..74b942e712 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -30,6 +30,7 @@ extern const char *progname;
 extern void set_dump_section(const char *arg, int *dumpSections);
 extern void on_exit_nicely(on_exit_nicely_callback function, void *arg);
 extern void exit_nicely(int code) pg_attribute_noreturn();
+extern void parse_filters_from_file(const char *filename, void *priv);
 
 /* In pg_dump, we modify pg_fatal to call exit_nicely instead of exit */
 #undef pg_fatal
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 67b6d9079e..d88adfe59a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -58,6 +58,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -389,6 +390,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -622,6 +624,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				parse_filters_from_file(optarg, (void *) &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1027,6 +1033,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18197,3 +18205,51 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 69ae027bd3..4e9d898387 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -79,6 +80,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -156,6 +158,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -358,6 +361,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -651,6 +659,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1888,7 +1897,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1912,3 +1920,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..70897d9056 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,6 +47,7 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
@@ -123,6 +124,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +288,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				parse_filters_from_file(optarg, (void *) opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +469,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +501,54 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 098bc3f1b0..6418db9261 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.32.1 (Apple Git-133)

#142Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#141)
Re: proposal: possibility to read dumped table's name from file

po 12. 9. 2022 v 9:59 odesílatel Daniel Gustafsson <daniel@yesql.se> napsal:

On 9 Sep 2022, at 11:00, Andrew Dunstan <andrew@dunslane.net> wrote:

On Sep 9, 2022, at 5:53 PM, John Naylor <john.naylor@enterprisedb.com>

wrote:

Note that the grammar has shift-reduce conflicts.

Looks like the last rule for Filters should not be there.

Correct, fixed in the attached.

I do wonder whether we should be using bison/flex here, seems like using

a

sledgehammer to crack a nut.

I don't the capabilities of the tool is all that interesting compared to
the
long term maintainability and readability of the source code. Personally I
think a simple Bison/Flex parser is easier to read and reason about than
the
corresponding written in C.

When this work is done, then there is no reason to throw it. The parser in
bison/flex does the same work and it is true, so code is more readable.
Although for this case, a handy written parser was trivial too.

Regards

Pavel

Show quoted text

--
Daniel Gustafsson https://vmware.com/

#143Erik Rijkers
er@xs4all.nl
In reply to: Daniel Gustafsson (#141)
Re: proposal: possibility to read dumped table's name from file

Op 12-09-2022 om 09:58 schreef Daniel Gustafsson:

On 9 Sep 2022, at 11:00, Andrew Dunstan <andrew@dunslane.net> wrote:

On Sep 9, 2022, at 5:53 PM, John Naylor <john.naylor@enterprisedb.com> wrote:

[v4-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch]

I noticed that pg_restore --filter cannot, or at last not always, be
used with the same filter-file that was used to produce a dump with
pg_dump --filter.

Is that as designed? It seems a bit counterintuitive. It'd be nice if
that could be fixed. Admittedly, the 'same' problem in pg_restore -t,
also less than ideal.

(A messy bashdemo below)

thanks,

Erik Rijkers

#! /bin/bash
db2='testdb2' db3='testdb3'
db2='testdb_source' db3='testdb_target'
sql_dropdb="drop database if exists $db2; drop database if exists $db3;"
sql_createdb="create database $db2; create database $db3;"
schema1=s1 table1=table1 t1=$schema1.$table1
schema2=s2 table2=table2 t2=$schema2.$table2
sql_schema_init="create schema if not exists $schema1; create schema if
not exists $schema2;"
sql_test="select '$t1', n from $t1 order by n; select '$t2', n from $t2
order by n;"

function sqltest()
{
for database_name in $db2 $db3 ;do
port_used=$( echo "show port" |psql -qtAX -d $database_name )
echo -n "-- $database_name ($port_used): "
echo "$sql_test" | psql -qtAX -a -d $database_name | md5sum
done
echo
}

echo "setting up orig db $db2, target db $db3"
echo "$sql_dropdb" | psql -qtAX
echo "$sql_createdb" | psql -qtAX

psql -X -d $db2 << SQL
$sql_schema_init
create table $t1 as select n from generate_series(1, (10^1)::int) as f(n);
create table $t2 as select n from generate_series(2, (10^2)::int) as f(n);
SQL
echo "
include table $t1
include table $t2
# include schema $s1
# include schema $s2
" > inputfile1.txt

# in filter; out plain
echo "-- pg_dump -F p -f plainfile1 --filter=inputfile1.txt -d $db2"
pg_dump -F p -f plainfile1 --filter=inputfile1.txt -d $db2

echo "$sql_schema_init" | psql -qX -d $db3
echo "-- pg_restore -d $db3 dumpfile1"
pg_restore -d $db3 dumpfile1
rc=$?
echo "-- pg_restore returned [$rc] -- pg_restore without --filter"
sqltest

# enable this to see it fail
if [[ 1 -eq 1 ]]
then

# clean out
echo "drop schema $schema1 cascade; drop schema $schema2 cascade; " |
psql -qtAXad $db3

--filter=inputfile1.txt"
echo "$sql_schema_init" | psql -qX -d $db3
echo "-- pg_restore -d $db3 --filter=inputfile1.txt dumpfile1"
pg_restore -d $db3 --filter=inputfile1.txt dumpfile1
rc=$?
echo "-- pg_restore returned [$rc] -- pg_restore without --filter"
sqltest

fi

#144Erik Rijkers
er@xs4all.nl
In reply to: Erik Rijkers (#143)
Re: proposal: possibility to read dumped table's name from file

Op 12-09-2022 om 16:00 schreef Erik Rijkers:

Op 12-09-2022 om 09:58 schreef Daniel Gustafsson:

On 9 Sep 2022, at 11:00, Andrew Dunstan <andrew@dunslane.net> wrote:

On Sep 9, 2022, at 5:53 PM, John Naylor
<john.naylor@enterprisedb.com> wrote:

[v4-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patch]

I noticed that pg_restore --filter cannot, or at last not always, be
used with the same filter-file that was used to produce a dump with
pg_dump --filter.

Is that as designed?  It seems a bit counterintuitive.  It'd be nice if
that could be fixed.  Admittedly, the 'same' problem in pg_restore -t,
also less than ideal.

(A messy bashdemo below)

I hope the issue is still clear, even though in the bash I sent, I
messed up the dumpfile name (i.e., in the bash that I sent the pg_dump
creates another dump name than what is given to pg_restore. They should
use the same dumpname, obviously)

Show quoted text

thanks,

Erik Rijkers

#145John Naylor
john.naylor@enterprisedb.com
In reply to: Pavel Stehule (#142)
Re: proposal: possibility to read dumped table's name from file

On Mon, Sep 12, 2022 at 8:10 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

po 12. 9. 2022 v 9:59 odesílatel Daniel Gustafsson <daniel@yesql.se> napsal:

I don't the capabilities of the tool is all that interesting compared to the
long term maintainability and readability of the source code.

With make distprep and maintainer-clean, separate makefile and MSVC
build logic a short time before converting to Meson, I'm not sure that
even the short term maintainability here is a good trade off for what
we're getting.

The parser in bison/flex does the same work and it is true, so code is more readable. Although for this case, a handy written parser was trivial too.

If the hand-written version is trivial, then we should prefer it.
--
John Naylor
EDB: http://www.enterprisedb.com

#146Pavel Stehule
pavel.stehule@gmail.com
In reply to: John Naylor (#145)
Re: proposal: possibility to read dumped table's name from file

út 13. 9. 2022 v 10:46 odesílatel John Naylor <john.naylor@enterprisedb.com>
napsal:

On Mon, Sep 12, 2022 at 8:10 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

po 12. 9. 2022 v 9:59 odesílatel Daniel Gustafsson <daniel@yesql.se>

napsal:

I don't the capabilities of the tool is all that interesting compared

to the

long term maintainability and readability of the source code.

With make distprep and maintainer-clean, separate makefile and MSVC
build logic a short time before converting to Meson, I'm not sure that
even the short term maintainability here is a good trade off for what
we're getting.

The parser in bison/flex does the same work and it is true, so code is

more readable. Although for this case, a handy written parser was trivial
too.

If the hand-written version is trivial, then we should prefer it.

Please, can you check and compare both versions? My view is subjective.

Regards

Pavel

Show quoted text

--
John Naylor
EDB: http://www.enterprisedb.com

#147Andres Freund
andres@anarazel.de
In reply to: Daniel Gustafsson (#141)
Re: proposal: possibility to read dumped table's name from file

Hi,

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

On 9 Sep 2022, at 11:00, Andrew Dunstan <andrew@dunslane.net> wrote:

On Sep 9, 2022, at 5:53 PM, John Naylor <john.naylor@enterprisedb.com> wrote:

Note that the grammar has shift-reduce conflicts.

Looks like the last rule for Filters should not be there.

Correct, fixed in the attached.

Due to the merge of the meson build, this patch now needs to adjust the
relevant meson.build. This is the cause of the failures at:
https://cirrus-ci.com/build/5788292678418432

See e.g. src/bin/pgbench/meson.build

Greetings,

Andres Freund

#148Andres Freund
andres@anarazel.de
In reply to: Daniel Gustafsson (#141)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi,

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

Correct, fixed in the attached.

Updated patch adding meson compatibility attached.

Greetings,

Andres Freund

Attachments:

v5-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchtext/x-diff; charset=us-asciiDownload
From 5d3ba4d2d6567626ccc0019208ea4c0ea91ac866 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH v5] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 src/bin/pg_dump/.gitignore        |   4 +
 src/bin/pg_dump/Makefile          |  17 +++-
 src/bin/pg_dump/filter.h          |  44 ++++++++++
 src/bin/pg_dump/filterparse.y     |  64 ++++++++++++++
 src/bin/pg_dump/filterscan.l      | 139 ++++++++++++++++++++++++++++++
 src/bin/pg_dump/meson.build       |  18 ++++
 src/bin/pg_dump/pg_backup_utils.c |  33 +++++++
 src/bin/pg_dump/pg_backup_utils.h |   1 +
 src/bin/pg_dump/pg_dump.c         |  56 ++++++++++++
 src/bin/pg_dump/pg_dumpall.c      |  65 +++++++++++++-
 src/bin/pg_dump/pg_restore.c      |  58 +++++++++++++
 doc/src/sgml/ref/pg_dump.sgml     |  88 +++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml  |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml  |  25 ++++++
 src/tools/msvc/Mkvcbuild.pm       |   4 +-
 15 files changed, 634 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d78127793..11f2d68bea0 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd2..e3befdc9b1f 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 00000000000..5dff4161f02
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 00000000000..c7f06f788d3
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,64 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Filters
+
+%%
+
+Filters:
+	/* EMPTY */
+	| Filters Filter
+	;
+
+Filter:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 00000000000..0d7341086ba
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,139 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>({space}*)data			{ BEGIN(type); return OBJ_DATA; }
+<command>({space}*)database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>({space}*)foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>({space}*)function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>({space}*)index			{ BEGIN(type); return OBJ_INDEX; }
+<command>({space}*)schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>({space}*)table			{ BEGIN(type); return OBJ_TABLE; }
+<command>({space}*)trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{space}		{ /* ignore */ }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* Ensure that we don't overflow */
+		if (patternbufsize > (SIZE_MAX / 2))
+		{
+			fprintf(stderr, "out of memory\n");
+			exit(1);
+		}
+
+		/* Double the size of patternbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 785ec094dbd..62da27fd501 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -11,8 +11,26 @@ pg_dump_common_sources = files(
   'pg_backup_utils.c',
 )
 
+
+filterscan = custom_target('filterscan',
+  input: 'filterscan.l',
+  output: 'filterscan.c',
+  command: flex_cmd,
+)
+generated_sources += filterscan
+pg_dump_common_sources += filterscan
+
+filterparse = custom_target('',
+  input: 'filterparse.y',
+  output: 'filterparse.c',
+  command: bison_cmd,
+)
+generated_sources += filterparse
+pg_dump_common_sources += filterparse
+
 pg_dump_common = static_library('libpgdump_common',
   pg_dump_common_sources,
+  include_directories: '.',
   dependencies: [frontend_code, libpq, zlib],
   kwargs: internal_lib_args,
 )
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index e40890cb264..d6db156bae8 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,6 +13,7 @@
  */
 #include "postgres_fe.h"
 
+#include "filter.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
@@ -102,3 +103,35 @@ exit_nicely(int code)
 
 	exit(code);
 }
+
+/*
+ * Retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+void
+parse_filters_from_file(const char *filename, void *priv)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse(priv);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 8173bb93cfc..74b942e712c 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -30,6 +30,7 @@ extern const char *progname;
 extern void set_dump_section(const char *arg, int *dumpSections);
 extern void on_exit_nicely(on_exit_nicely_callback function, void *arg);
 extern void exit_nicely(int code) pg_attribute_noreturn();
+extern void parse_filters_from_file(const char *filename, void *priv);
 
 /* In pg_dump, we modify pg_fatal to call exit_nicely instead of exit */
 #undef pg_fatal
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bd9b066e4eb..da69cee0cf1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -390,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				parse_filters_from_file(optarg, (void *) &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18206,51 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39d..e2b0e241fd2 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +661,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1899,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1922,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a1006347..70897d90566 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,6 +47,7 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
@@ -123,6 +124,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +288,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				parse_filters_from_file(optarg, (void *) opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +469,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +501,54 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad4..955bfcfdad2 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab5..7d8de30310b 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda06..ffeb564c520 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index ddb4f25eb12..4ae0c94319f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.37.3.542.gdd3f6c4cae

#149Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#148)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi,

On 2022-10-01 23:56:59 -0700, Andres Freund wrote:

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

Correct, fixed in the attached.

Updated patch adding meson compatibility attached.

Err, forgot to amend one hunk :(

Greetings,

Andres Freund

Attachments:

v6-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchtext/x-diff; charset=us-asciiDownload
From fe2926ccd49a460cbaa39a8916a4dd097463b294 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH v6] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 src/bin/pg_dump/.gitignore        |   4 +
 src/bin/pg_dump/Makefile          |  17 +++-
 src/bin/pg_dump/filter.h          |  44 ++++++++++
 src/bin/pg_dump/filterparse.y     |  64 ++++++++++++++
 src/bin/pg_dump/filterscan.l      | 139 ++++++++++++++++++++++++++++++
 src/bin/pg_dump/meson.build       |  18 ++++
 src/bin/pg_dump/pg_backup_utils.c |  33 +++++++
 src/bin/pg_dump/pg_backup_utils.h |   1 +
 src/bin/pg_dump/pg_dump.c         |  56 ++++++++++++
 src/bin/pg_dump/pg_dumpall.c      |  65 +++++++++++++-
 src/bin/pg_dump/pg_restore.c      |  58 +++++++++++++
 doc/src/sgml/ref/pg_dump.sgml     |  88 +++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml  |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml  |  25 ++++++
 src/tools/msvc/Mkvcbuild.pm       |   4 +-
 15 files changed, 634 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d78127793..11f2d68bea0 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd2..e3befdc9b1f 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 00000000000..5dff4161f02
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 00000000000..c7f06f788d3
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,64 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Filters
+
+%%
+
+Filters:
+	/* EMPTY */
+	| Filters Filter
+	;
+
+Filter:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 00000000000..0d7341086ba
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,139 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>({space}*)data			{ BEGIN(type); return OBJ_DATA; }
+<command>({space}*)database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>({space}*)foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>({space}*)function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>({space}*)index			{ BEGIN(type); return OBJ_INDEX; }
+<command>({space}*)schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>({space}*)table			{ BEGIN(type); return OBJ_TABLE; }
+<command>({space}*)trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{space}		{ /* ignore */ }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* Ensure that we don't overflow */
+		if (patternbufsize > (SIZE_MAX / 2))
+		{
+			fprintf(stderr, "out of memory\n");
+			exit(1);
+		}
+
+		/* Double the size of patternbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 785ec094dbd..8ec15cd4534 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -11,8 +11,26 @@ pg_dump_common_sources = files(
   'pg_backup_utils.c',
 )
 
+
+filterscan = custom_target('filterscan',
+  input: 'filterscan.l',
+  output: 'filterscan.c',
+  command: flex_cmd,
+)
+generated_sources += filterscan
+pg_dump_common_sources += filterscan
+
+filterparse = custom_target('filterparse',
+  input: 'filterparse.y',
+  output: 'filterparse.c',
+  command: bison_cmd,
+)
+generated_sources += filterparse
+pg_dump_common_sources += filterparse
+
 pg_dump_common = static_library('libpgdump_common',
   pg_dump_common_sources,
+  include_directories: '.',
   dependencies: [frontend_code, libpq, zlib],
   kwargs: internal_lib_args,
 )
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index e40890cb264..d6db156bae8 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,6 +13,7 @@
  */
 #include "postgres_fe.h"
 
+#include "filter.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
@@ -102,3 +103,35 @@ exit_nicely(int code)
 
 	exit(code);
 }
+
+/*
+ * Retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+void
+parse_filters_from_file(const char *filename, void *priv)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse(priv);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 8173bb93cfc..74b942e712c 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -30,6 +30,7 @@ extern const char *progname;
 extern void set_dump_section(const char *arg, int *dumpSections);
 extern void on_exit_nicely(on_exit_nicely_callback function, void *arg);
 extern void exit_nicely(int code) pg_attribute_noreturn();
+extern void parse_filters_from_file(const char *filename, void *priv);
 
 /* In pg_dump, we modify pg_fatal to call exit_nicely instead of exit */
 #undef pg_fatal
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bd9b066e4eb..da69cee0cf1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -390,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				parse_filters_from_file(optarg, (void *) &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18206,51 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39d..e2b0e241fd2 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +661,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1899,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1922,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a1006347..70897d90566 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,6 +47,7 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
@@ -123,6 +124,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +288,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				parse_filters_from_file(optarg, (void *) opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +469,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +501,54 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad4..955bfcfdad2 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab5..7d8de30310b 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda06..ffeb564c520 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index ddb4f25eb12..4ae0c94319f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.37.3.542.gdd3f6c4cae

#150Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#149)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi,

On 2022-10-02 00:19:59 -0700, Andres Freund wrote:

On 2022-10-01 23:56:59 -0700, Andres Freund wrote:

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

Correct, fixed in the attached.

Updated patch adding meson compatibility attached.

Err, forgot to amend one hunk :(

That fixed it on all platforms but windows, due to copy-pasto. I really should
have stopped earlier yesterday...

+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"

c.h (and postgres.h, postgres_fe.h) shouldn't be included in headers.

This is a common enough mistake that I'm wondering if we could automate
warning about it somehow.

Greetings,

Andres Freund

Attachments:

v7-0001-Add-include-exclude-filtering-via-file-in-pg_dump.patchtext/x-diff; charset=us-asciiDownload
From e0dff3c3275a69b53fdf1c628c204365bb96852f Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 7 Sep 2022 15:22:05 +0200
Subject: [PATCH v7] Add include/exclude filtering via file in pg_dump

Author: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 src/bin/pg_dump/.gitignore        |   4 +
 src/bin/pg_dump/Makefile          |  17 +++-
 src/bin/pg_dump/filter.h          |  44 ++++++++++
 src/bin/pg_dump/filterparse.y     |  64 ++++++++++++++
 src/bin/pg_dump/filterscan.l      | 139 ++++++++++++++++++++++++++++++
 src/bin/pg_dump/meson.build       |  17 ++++
 src/bin/pg_dump/pg_backup_utils.c |  33 +++++++
 src/bin/pg_dump/pg_backup_utils.h |   1 +
 src/bin/pg_dump/pg_dump.c         |  56 ++++++++++++
 src/bin/pg_dump/pg_dumpall.c      |  65 +++++++++++++-
 src/bin/pg_dump/pg_restore.c      |  58 +++++++++++++
 doc/src/sgml/ref/pg_dump.sgml     |  88 +++++++++++++++++++
 doc/src/sgml/ref/pg_dumpall.sgml  |  22 +++++
 doc/src/sgml/ref/pg_restore.sgml  |  25 ++++++
 src/tools/msvc/Mkvcbuild.pm       |   4 +-
 15 files changed, 633 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/filterparse.y
 create mode 100644 src/bin/pg_dump/filterscan.l

diff --git a/src/bin/pg_dump/.gitignore b/src/bin/pg_dump/.gitignore
index e6d78127793..11f2d68bea0 100644
--- a/src/bin/pg_dump/.gitignore
+++ b/src/bin/pg_dump/.gitignore
@@ -2,4 +2,8 @@
 /pg_dumpall
 /pg_restore
 
+# Local generated source files
+/filterparse.c
+/filterscan.c
+
 /tmp_check/
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd2..e3befdc9b1f 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,8 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filterparse.o \
+	filterscan.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -37,14 +39,23 @@ OBJS = \
 
 all: pg_dump pg_restore pg_dumpall
 
+# See notes in src/backend/parser/Makefile about the following two rules
+filterparse.h: filterparse.c
+	touch $@
+
+filterparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+filterparse.o filterscan.o: filterparse.h
+
 pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_dump.o common.o pg_dump_sort.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filterparse.o filterscan.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
@@ -63,6 +74,8 @@ installcheck:
 uninstall:
 	rm -f $(addprefix '$(DESTDIR)$(bindir)'/, pg_dump$(X) pg_restore$(X) pg_dumpall$(X))
 
+distprep: filterparse.c filterscan.c
+
 clean distclean maintainer-clean:
 	rm -f pg_dump$(X) pg_restore$(X) pg_dumpall$(X) $(OBJS) pg_dump.o common.o pg_dump_sort.o pg_restore.o pg_dumpall.o
 	rm -rf tmp_check
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 00000000000..5dff4161f02
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern FILE *filter_yyin;
+
+extern int filter_yylex(void);
+extern void filter_yyerror(void *priv, const char *msg);
+extern void filter_scanner_init(void);
+extern void filter_scanner_finish(void);
+extern int filter_yyparse(void *priv);
+
+extern void include_item(void *priv, FilterObjectType objtype, const char *objname);
+extern void exclude_item(void *priv, FilterObjectType objtype, const char *objname);
+
+#endif
diff --git a/src/bin/pg_dump/filterparse.y b/src/bin/pg_dump/filterparse.y
new file mode 100644
index 00000000000..c7f06f788d3
--- /dev/null
+++ b/src/bin/pg_dump/filterparse.y
@@ -0,0 +1,64 @@
+%{
+/*-------------------------------------------------------------------------
+ *
+ * filterparse.y
+ *	  bison grammar for the pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterparse.y
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+%}
+
+%name-prefix="filter_yy"
+%parse-param {void *priv}
+
+%union
+{
+	char	   *str;
+	int			integer;
+}
+
+%type	<integer> include_object exclude_object
+%token	<str> pattern
+%token	C_INCLUDE C_EXCLUDE
+%token	OBJ_DATA OBJ_DATABASE OBJ_FOREIGN_DATA OBJ_FUNCTION OBJ_INDEX
+%token	OBJ_SCHEMA OBJ_TABLE OBJ_TRIGGER
+%start	Filters
+
+%%
+
+Filters:
+	/* EMPTY */
+	| Filters Filter
+	;
+
+Filter:
+		C_INCLUDE include_object pattern { include_item(priv, $2, $3); }
+		| C_EXCLUDE exclude_object pattern { exclude_item(priv, $2, $3); }
+		;
+
+include_object:
+		OBJ_FOREIGN_DATA		{ $$ = FILTER_OBJECT_TYPE_FOREIGN_DATA; }
+		| OBJ_FUNCTION			{ $$ = FILTER_OBJECT_TYPE_FUNCTION; }
+		| OBJ_INDEX				{ $$ = FILTER_OBJECT_TYPE_INDEX; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		| OBJ_TRIGGER			{ $$ = FILTER_OBJECT_TYPE_TRIGGER; }
+		;
+
+exclude_object:
+		OBJ_DATA				{ $$ = FILTER_OBJECT_TYPE_DATA; }
+		| OBJ_DATABASE			{ $$ = FILTER_OBJECT_TYPE_DATABASE; }
+		| OBJ_SCHEMA			{ $$ = FILTER_OBJECT_TYPE_SCHEMA; }
+		| OBJ_TABLE				{ $$ = FILTER_OBJECT_TYPE_TABLE; }
+		;
+
+%%
diff --git a/src/bin/pg_dump/filterscan.l b/src/bin/pg_dump/filterscan.l
new file mode 100644
index 00000000000..0d7341086ba
--- /dev/null
+++ b/src/bin/pg_dump/filterscan.l
@@ -0,0 +1,139 @@
+%top{
+/*-------------------------------------------------------------------------
+ *
+ * filterscan.l
+ *	  a lexical scanner for pg_dump object filter files
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filterscan.l
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+#include "filterparse.h"
+#include "filter.h"
+}
+
+%{
+
+static int yyline = 1;
+
+static void add_pattern_char(char c);
+
+#define PATTERNBUF_INIT 128
+
+static char	   *patternbuf = NULL;
+static size_t	patternbufsize = PATTERNBUF_INIT;
+static size_t	patternbufpos = 0;
+
+%}
+
+%option never-interactive
+%option noinput
+%option nounput
+%option noyywrap
+%option warn
+%option prefix="filter_yy"
+
+%x command
+%x type
+%x qident
+
+space			[ \t\r\f\v]
+newline			[\n]
+non_newline		[^\n\r]
+
+comment			("#"{non_newline}*)
+pattern			([^"\n# \t\r\f\v]+)
+
+%%
+
+ /* Commands */
+include			{ BEGIN(command); return C_INCLUDE; }
+exclude			{ BEGIN(command); return C_EXCLUDE; }
+
+[\n]			yyline++; 
+{comment}		{ /* ignore */ }
+{space}			{ /* ignore */ }
+
+ /* Object types */
+<command>({space}*)data			{ BEGIN(type); return OBJ_DATA; }
+<command>({space}*)database		{ BEGIN(type); return OBJ_DATABASE; }
+<command>({space}*)foreign_data	{ BEGIN(type); return OBJ_FOREIGN_DATA; }
+<command>({space}*)function		{ BEGIN(type); return OBJ_FUNCTION; }
+<command>({space}*)index			{ BEGIN(type); return OBJ_INDEX; }
+<command>({space}*)schema			{ BEGIN(type); return OBJ_SCHEMA; }
+<command>({space}*)table			{ BEGIN(type); return OBJ_TABLE; }
+<command>({space}*)trigger		{ BEGIN(type); return OBJ_TRIGGER; }
+<command>{pattern}		{ filter_yyerror(NULL, "unsupported filter object type"); }
+<command><<EOF>>		{ filter_yyerror(NULL, "missing type name"); }
+
+ /* Patterns */
+<type><<EOF>>		{ filter_yyerror(NULL, "missing object name"); }
+<type>{space}		{ /* ignore */ }
+<type>{pattern}		{
+						filter_yylval.str = pg_strdup(yytext);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<type>\"			{
+						patternbufpos = 0;
+						memset(patternbuf, 0, patternbufsize);
+						BEGIN(qident);
+					}
+<qident>\\\"		{ add_pattern_char(yytext[1]); }
+<qident>\"			{
+						filter_yylval.str = pg_strdup(patternbuf);
+						BEGIN(INITIAL);
+						return(pattern);
+					}
+<qident>{newline}	{ add_pattern_char('\n'); }
+<qident>.			{ add_pattern_char(yytext[0]); }
+<qident><<EOF>>		{
+						fprintf(stderr, "syntax error at line %i; unterminated quoted pattern: %s\n",
+								yyline, patternbuf);
+						exit(1);
+					}
+ /* Anything else is a syntax error */
+.					{ filter_yyerror(NULL, "unexpected character"); }
+%%
+
+static void
+add_pattern_char(char c)
+{
+	/* Leave room for trailing zero */
+	if (patternbufpos >= patternbufsize - 1)
+	{
+		/* Ensure that we don't overflow */
+		if (patternbufsize > (SIZE_MAX / 2))
+		{
+			fprintf(stderr, "out of memory\n");
+			exit(1);
+		}
+
+		/* Double the size of patternbuf if it gets full */
+		patternbufsize += patternbufsize;
+		patternbuf = pg_realloc(patternbuf, patternbufsize);
+	}
+	patternbuf[patternbufpos++] = c;
+}
+
+void
+filter_scanner_init(void)
+{
+	patternbuf = pg_malloc(patternbufsize);
+}
+
+void
+filter_scanner_finish(void)
+{
+	pfree(patternbuf);
+}
+
+void
+filter_yyerror(void *priv, const char *msg)
+{
+	fprintf(stderr, "syntax error at line %i: %s: \"%s\"\n", yyline, msg, yytext);
+	exit(1);
+}
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 785ec094dbd..23ca8c3e890 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -11,8 +11,25 @@ pg_dump_common_sources = files(
   'pg_backup_utils.c',
 )
 
+
+filterscan = custom_target('filterscan',
+  input: 'filterscan.l',
+  output: 'filterscan.c',
+  command: flex_cmd,
+)
+generated_sources += filterscan
+pg_dump_common_sources += filterscan
+
+filterparse = custom_target('filterparse',
+  input: 'filterparse.y',
+  kwargs: bison_kw,
+)
+generated_sources += filterparse.to_list()
+pg_dump_common_sources += filterparse
+
 pg_dump_common = static_library('libpgdump_common',
   pg_dump_common_sources,
+  include_directories: '.',
   dependencies: [frontend_code, libpq, zlib],
   kwargs: internal_lib_args,
 )
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index e40890cb264..d6db156bae8 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,6 +13,7 @@
  */
 #include "postgres_fe.h"
 
+#include "filter.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
@@ -102,3 +103,35 @@ exit_nicely(int code)
 
 	exit(code);
 }
+
+/*
+ * Retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+void
+parse_filters_from_file(const char *filename, void *priv)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse(priv);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 8173bb93cfc..74b942e712c 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -30,6 +30,7 @@ extern const char *progname;
 extern void set_dump_section(const char *arg, int *dumpSections);
 extern void on_exit_nicely(on_exit_nicely_callback function, void *arg);
 extern void exit_nicely(int code) pg_attribute_noreturn();
+extern void parse_filters_from_file(const char *filename, void *priv);
 
 /* In pg_dump, we modify pg_fatal to call exit_nicely instead of exit */
 #undef pg_fatal
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bd9b066e4eb..da69cee0cf1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -390,6 +391,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +625,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				parse_filters_from_file(optarg, (void *) &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1034,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18206,51 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_DATA:
+			simple_string_list_append(&tabledata_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_exclude_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_exclude_patterns, objname);
+			break;
+		default:
+			pg_log_error("Unsupported exclude object");
+			exit_nicely(1);
+			break;
+	}
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	DumpOptions *dopt = (DumpOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			simple_string_list_append(&foreign_servers_include_patterns, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&schema_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			simple_string_list_append(&table_include_patterns, objname);
+			dopt->include_everything = false;
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39d..e2b0e241fd2 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,11 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				getDatabaseExcludeFiltersFromFile(optarg,
+												  &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +661,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1899,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1922,58 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * getDatabaseFiltersFromFile - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+getDatabaseExcludeFiltersFromFile(const char *filename, SimpleStringList *pattern)
+{
+	bool need_close = true;
+
+	if (strlen(filename) == 1 && filename[0] == '-')
+	{
+		filter_yyin = stdin;
+		need_close = false;
+	}
+	else
+	{
+		filter_yyin = fopen(filename, "r");
+		if (!filter_yyin)
+			pg_fatal("unable to open filterfile \"%s\": %m", filename);
+	}
+
+	filter_scanner_init();
+	filter_yyparse((void *) pattern);
+	filter_scanner_finish();
+
+	if (need_close)
+		fclose(filter_yyin);
+}
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	SimpleStringList *pattern = (SimpleStringList *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_DATABASE)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(pattern, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	(void) priv;
+
+	pg_log_error("Unsupported include object");
+	exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a1006347..70897d90566 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,6 +47,7 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
@@ -123,6 +124,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +288,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				parse_filters_from_file(optarg, (void *) opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +469,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +501,54 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+void
+exclude_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	if (objtype != FILTER_OBJECT_TYPE_SCHEMA)
+	{
+		pg_log_error("Unsupported exclude object");
+		exit_nicely(1);
+	}
+
+	simple_string_list_append(&opts->schemaExcludeNames, objname);
+}
+
+void
+include_item(void *priv, FilterObjectType objtype, const char *objname)
+{
+	RestoreOptions *opts = (RestoreOptions *) priv;
+
+	switch (objtype)
+	{
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&opts->functionNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_INDEX:
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&opts->indexNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			simple_string_list_append(&opts->schemaNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TABLE:
+			opts->selTypes = 1;
+			opts->selTable = 1;
+			simple_string_list_append(&opts->tableNames, objname);
+			break;
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&opts->triggerNames, optarg);
+			break;
+		default:
+			pg_log_error("Unsupported include object");
+			exit_nicely(1);
+			break;
+	}
+}
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad4..955bfcfdad2 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab5..7d8de30310b 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda06..ffeb564c520 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index ddb4f25eb12..4ae0c94319f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -78,7 +78,9 @@ my $frontend_extraincludes = {
 my $frontend_extrasource = {
 	'psql' => ['src/bin/psql/psqlscanslash.l'],
 	'pgbench' =>
-	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ]
+	  [ 'src/bin/pgbench/exprscan.l', 'src/bin/pgbench/exprparse.y' ],
+	'pg_dump' =>
+	  ['src/bin/pg_dump/filterscan.l', 'src/bin/pg_dump/filterparse.y']
 };
 my @frontend_excludes = (
 	'pgevent',    'pg_basebackup', 'pg_rewind', 'pg_dump',
-- 
2.37.3.542.gdd3f6c4cae

#151Daniel Gustafsson
daniel@yesql.se
In reply to: Andres Freund (#150)
Re: proposal: possibility to read dumped table's name from file

On 2 Oct 2022, at 18:04, Andres Freund <andres@anarazel.de> wrote:
On 2022-10-02 00:19:59 -0700, Andres Freund wrote:

On 2022-10-01 23:56:59 -0700, Andres Freund wrote:

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

Correct, fixed in the attached.

Updated patch adding meson compatibility attached.

Err, forgot to amend one hunk :(

That fixed it on all platforms but windows, due to copy-pasto. I really should
have stopped earlier yesterday...

Thanks for updating the patch!

The parser in the original submission was -1'd by me, and the current version
proposed as an alternative. This was subsequently -1'd as well but no updated
patch with a rewritten parser has been posted. So this is now stalled again.

Having been around in 12 commitfests without a committer feeling confident
about pushing this I plan to mark it returned with feedback, and if a new
parser materializes itc can be readded instead of being dragged along.

c.h (and postgres.h, postgres_fe.h) shouldn't be included in headers.

This is a common enough mistake that I'm wondering if we could automate
warning about it somehow.

Maybe we can add a simple git grep invocation in the CompilerWarnings CI job to
catch this in the CFBot? If something like the below sketch matches then we
can throw an error. (only for illustration, all three files needs to checked).

git grep "\"c\.h" -- *.h :^src/include/postgres*.h;

--
Daniel Gustafsson https://vmware.com/

#152Tom Lane
tgl@sss.pgh.pa.us
In reply to: Daniel Gustafsson (#151)
Re: proposal: possibility to read dumped table's name from file

Daniel Gustafsson <daniel@yesql.se> writes:

On 2 Oct 2022, at 18:04, Andres Freund <andres@anarazel.de> wrote:

c.h (and postgres.h, postgres_fe.h) shouldn't be included in headers.
This is a common enough mistake that I'm wondering if we could automate
warning about it somehow.

Maybe we can add a simple git grep invocation in the CompilerWarnings CI job to
catch this in the CFBot?

I'd be inclined to teach headerscheck or cpluspluscheck about it.

regards, tom lane

#153Andres Freund
andres@anarazel.de
In reply to: Daniel Gustafsson (#151)
Re: proposal: possibility to read dumped table's name from file

On 2022-10-02 22:52:33 +0200, Daniel Gustafsson wrote:

The parser in the original submission was -1'd by me, and the current version
proposed as an alternative. This was subsequently -1'd as well but no updated
patch with a rewritten parser has been posted. So this is now stalled again.

Having been around in 12 commitfests without a committer feeling confident
about pushing this I plan to mark it returned with feedback, and if a new
parser materializes itc can be readded instead of being dragged along.

Makes sense to me.

#154Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#151)
Re: proposal: possibility to read dumped table's name from file

ne 2. 10. 2022 v 22:52 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 2 Oct 2022, at 18:04, Andres Freund <andres@anarazel.de> wrote:
On 2022-10-02 00:19:59 -0700, Andres Freund wrote:

On 2022-10-01 23:56:59 -0700, Andres Freund wrote:

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

Correct, fixed in the attached.

Updated patch adding meson compatibility attached.

Err, forgot to amend one hunk :(

That fixed it on all platforms but windows, due to copy-pasto. I really

should

have stopped earlier yesterday...

Thanks for updating the patch!

The parser in the original submission was -1'd by me, and the current
version
proposed as an alternative. This was subsequently -1'd as well but no
updated
patch with a rewritten parser has been posted. So this is now stalled
again.

You started rewriting it, but you didn't finish it.

Unfortunately, there is not a clean opinion on using bison's parser for
this purpose. I understand that the complexity of this language is too low,
so the benefit of using bison's gramatic is low too. Personally, I have not
any problem using bison for this purpose. For this case, I think we compare
two similarly long ways, but unfortunately, customers that have a problem
with long command lines still have this problem.

Can we go forward? Daniel is strongly against handwritten parser. Is there
somebody strongly against bison's based parser? There is not any other way.

I am able to complete Daniel's patch, if there will not be objections.

Regards

Pavel

Show quoted text

Having been around in 12 commitfests without a committer feeling confident
about pushing this I plan to mark it returned with feedback, and if a new
parser materializes itc can be readded instead of being dragged along.

c.h (and postgres.h, postgres_fe.h) shouldn't be included in headers.

This is a common enough mistake that I'm wondering if we could automate
warning about it somehow.

Maybe we can add a simple git grep invocation in the CompilerWarnings CI
job to
catch this in the CFBot? If something like the below sketch matches then
we
can throw an error. (only for illustration, all three files needs to
checked).

git grep "\"c\.h" -- *.h :^src/include/postgres*.h;

--
Daniel Gustafsson https://vmware.com/

#155Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#154)
Re: proposal: possibility to read dumped table's name from file

Hi,

On Mon, Oct 03, 2022 at 06:00:12AM +0200, Pavel Stehule wrote:

ne 2. 10. 2022 v 22:52 odes�latel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 2 Oct 2022, at 18:04, Andres Freund <andres@anarazel.de> wrote:
On 2022-10-02 00:19:59 -0700, Andres Freund wrote:

On 2022-10-01 23:56:59 -0700, Andres Freund wrote:

On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote:

Correct, fixed in the attached.

Updated patch adding meson compatibility attached.

Err, forgot to amend one hunk :(

That fixed it on all platforms but windows, due to copy-pasto. I really

should

have stopped earlier yesterday...

Thanks for updating the patch!

The parser in the original submission was -1'd by me, and the current
version
proposed as an alternative. This was subsequently -1'd as well but no
updated
patch with a rewritten parser has been posted. So this is now stalled
again.

You started rewriting it, but you didn't finish it.

Unfortunately, there is not a clean opinion on using bison's parser for
this purpose. I understand that the complexity of this language is too low,
so the benefit of using bison's gramatic is low too. Personally, I have not
any problem using bison for this purpose. For this case, I think we compare
two similarly long ways, but unfortunately, customers that have a problem
with long command lines still have this problem.

Can we go forward? Daniel is strongly against handwritten parser. Is there
somebody strongly against bison's based parser? There is not any other way.

I don't have a strong opinion either, but it seems that 2 people argued against
a bison parser (vs only 1 arguing for) and the fact that the current habit is
to rely on hand written parsers for simple cases (e.g. jsonapi.c /
pg_parse_json()), it seems that we should go back to Pavel's original parser.

I only had a quick look but it indeed seems trivial, it just maybe need a bit
of refactoring to avoid some code duplication (getFiltersFromFile is
duplicated, and getDatabaseExcludeFiltersFromFile could be removed if
getFiltersFromFile knew about the 2 patterns).

#156Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#155)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

I am sending version with handy written parser and meson support

po 3. 10. 2022 v 6:34 odesílatel Julien Rouhaud <rjuju123@gmail.com> napsal:

Hi,

You started rewriting it, but you didn't finish it.

Unfortunately, there is not a clean opinion on using bison's parser for
this purpose. I understand that the complexity of this language is too

low,

so the benefit of using bison's gramatic is low too. Personally, I have

not

any problem using bison for this purpose. For this case, I think we

compare

two similarly long ways, but unfortunately, customers that have a problem
with long command lines still have this problem.

Can we go forward? Daniel is strongly against handwritten parser. Is

there

somebody strongly against bison's based parser? There is not any other

way.

I don't have a strong opinion either, but it seems that 2 people argued
against
a bison parser (vs only 1 arguing for) and the fact that the current habit
is
to rely on hand written parsers for simple cases (e.g. jsonapi.c /
pg_parse_json()), it seems that we should go back to Pavel's original
parser.

I only had a quick look but it indeed seems trivial, it just maybe need a
bit
of refactoring to avoid some code duplication (getFiltersFromFile is
duplicated, and getDatabaseExcludeFiltersFromFile could be removed if
getFiltersFromFile knew about the 2 patterns).

I checked this code again, and I don't think some refactoring is easy.
getFiltersFromFile is not duplicated. It is just probably badly named.

These routines are used from pg_dump, pg_dumpall and pg_restore. There are
significant differences in supported objects and in types used for returned
lists (dumpOptions, SimpleStringList, and RestoreOptions). If I have one
routine, then I need to implement some mechanism for specification of
supported objects, and a special type that can be used as a proxy between
caller and parser to hold lists of parsed values. To be names less
confusing I renamed them to read_dump_filters, read_dumpall_filters and
read_restore_filters

Regards

Pavel

Attachments:

pg_dump-filter-20221007.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221007.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..955bfcfdad 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..5a76fd57f1
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,431 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"
+#include "common/fe_memutils.h"
+#include "common/string.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == bytes) && (pg_strncasecmp(cstr, str, bytes) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this rutine in pg_dumpall. So instead
+ * direct calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.
+ */
+
+/*
+ * Simple routines - just don't repeat same code
+ *
+ * Returns true, when filter's file is opened
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated sources for filter
+ */
+void
+filter_free_sources(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * log_format_error - Emit error message
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_DATA:
+			return "data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	return "unknown object type";
+}
+
+/*
+ * Helper routine to reduce duplicated code
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "The application \"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern. Returns pointer to the
+ * first character after the pattern.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	/*
+	 * If the object name pattern has been quoted, we must take care to parse
+	 * out the entire quoted pattern, which may contain whitespace and can span
+	 * many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						fstate->is_error = true;
+					}
+					else
+						log_invalid_filter_format(fstate, "unexpected end of file");
+
+					return NULL;
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * read_filter_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of:
+ * "table", "schema", "foreign_data", "data", "database", "function",
+ * "trigger" or "index". The pattern is either simple without any
+ * whitespace, or properly quoted in case there may be whitespace in the
+ * object name. The pattern handling follows the same rules as other object
+ * include and exclude functions; it can use wildcards. Returns true, when
+ * one filter item was successfully read and parsed.  When object name
+ * contains \n chars, then more than one line from input file can be
+ * processed. Returns false when the filter file reaches EOF. In case of
+ * errors, the function wont return but will exit with an appropriate error
+ * message.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			str = filter_get_pattern(fstate, str, objname);
+			if (!str)
+				return false;
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			while (isspace(*str))
+				str++;
+
+			if (*str != '\0' && *str != '#')
+			{
+				log_invalid_filter_format(fstate,
+										  "unexpected extra data after pattern");
+				return false;
+			}
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..f2503286aa
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free_sources(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bd9b066e4e..a5d958c3dc 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "include filter is not allowed for this type of object");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39..f393bb5eaf 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1898,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1921,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "include database filter is not suppported");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..701ca07a02 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, optarg);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free_sources(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..dd533d79eb
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,491 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 62;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_(one|two|three|three_one)/m,
+	"tables dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/The application "pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted tables are not restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#157Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#156)
Re: proposal: possibility to read dumped table's name from file

On Fri, Oct 07, 2022 at 07:26:08AM +0200, Pavel Stehule wrote:

I checked this code again, and I don't think some refactoring is easy.
getFiltersFromFile is not duplicated. It is just probably badly named.

These routines are used from pg_dump, pg_dumpall and pg_restore. There are
significant differences in supported objects and in types used for returned
lists (dumpOptions, SimpleStringList, and RestoreOptions). If I have one
routine, then I need to implement some mechanism for specification of
supported objects, and a special type that can be used as a proxy between
caller and parser to hold lists of parsed values. To be names less
confusing I renamed them to read_dump_filters, read_dumpall_filters and
read_restore_filters

Ah right, I missed the different argument types. Now that the functions have
improved names it looks way clearer, and it seems just fine!

#158Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#157)
Re: proposal: possibility to read dumped table's name from file

pá 7. 10. 2022 v 16:03 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

On Fri, Oct 07, 2022 at 07:26:08AM +0200, Pavel Stehule wrote:

I checked this code again, and I don't think some refactoring is easy.
getFiltersFromFile is not duplicated. It is just probably badly named.

These routines are used from pg_dump, pg_dumpall and pg_restore. There

are

significant differences in supported objects and in types used for

returned

lists (dumpOptions, SimpleStringList, and RestoreOptions). If I have one
routine, then I need to implement some mechanism for specification of
supported objects, and a special type that can be used as a proxy between
caller and parser to hold lists of parsed values. To be names less
confusing I renamed them to read_dump_filters, read_dumpall_filters and
read_restore_filters

Ah right, I missed the different argument types. Now that the functions
have
improved names it looks way clearer, and it seems just fine!

Thank you for check

Pavel

#159Andres Freund
andres@anarazel.de
In reply to: Pavel Stehule (#156)
Re: proposal: possibility to read dumped table's name from file

Hi,

On 2022-10-07 07:26:08 +0200, Pavel Stehule wrote:

I am sending version with handy written parser and meson support

Given this is a new approach it seems inaccurate to have the CF entry marked
ready-for-committer. I've updated it to needs-review.

Greetings,

Andres Freund

#160Julien Rouhaud
rjuju123@gmail.com
In reply to: Andres Freund (#159)
Re: proposal: possibility to read dumped table's name from file

On Thu, Oct 13, 2022 at 11:46:34AM -0700, Andres Freund wrote:

Hi,

On 2022-10-07 07:26:08 +0200, Pavel Stehule wrote:

I am sending version with handy written parser and meson support

Given this is a new approach it seems inaccurate to have the CF entry marked
ready-for-committer. I've updated it to needs-review.

I just had a quick look at the rest of the patch.

For the parser, it seems that filter_get_pattern is reimplementing an
identifier parsing function but isn't entirely correct. It can correctly parse
quoted non-qualified identifiers and non-quoted qualified identifiers, but not
quoted and qualified ones. For instance:

$ echo 'include table nsp.tbl' | pg_dump --filter - >/dev/null
$echo $?
0

$ echo 'include table "TBL"' | pg_dump --filter - >/dev/null
$echo $?
0

$ echo 'include table "NSP"."TBL"' | pg_dump --filter - >/dev/null
pg_dump: error: invalid format of filter on line 1: unexpected extra data after pattern

This should also be covered in the regression tests.

I'm wondering if psql's parse_identifier() could be exported and reused here
rather than creating yet another version.

Nitpicking: the comments needs some improvements:

+ /*
+  * Simple routines - just don't repeat same code
+  *
+  * Returns true, when filter's file is opened
+  */
+ bool
+ filter_init(FilterStateData *fstate, const char *filename)

also, is there any reason why this function doesn't call exit_nicely in case of
error rather than letting each caller do it without any other cleanup?

+ /*
+  * Release allocated sources for filter
+  */
+ void
+ filter_free_sources(FilterStateData *fstate)

I'm assuming "ressources" not "sources"?

+ /*
+  * log_format_error - Emit error message
+  *
+  * This is mostly a convenience routine to avoid duplicating file closing code
+  * in multiple callsites.
+  */
+ void
+ log_invalid_filter_format(FilterStateData *fstate, char *message)

mismatch between comment and function name (same for filter_read_item)

+ static const char *
+ filter_object_type_name(FilterObjectType fot)

No description.

/*
* Helper routine to reduce duplicated code
*/
void
log_unsupported_filter_object_type(FilterStateData *fstate,
const char *appname,
FilterObjectType fot)

Need more helpful comment.

#161Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#160)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

út 18. 10. 2022 v 11:33 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

On Thu, Oct 13, 2022 at 11:46:34AM -0700, Andres Freund wrote:

Hi,

On 2022-10-07 07:26:08 +0200, Pavel Stehule wrote:

I am sending version with handy written parser and meson support

Given this is a new approach it seems inaccurate to have the CF entry

marked

ready-for-committer. I've updated it to needs-review.

I just had a quick look at the rest of the patch.

For the parser, it seems that filter_get_pattern is reimplementing an
identifier parsing function but isn't entirely correct. It can correctly
parse
quoted non-qualified identifiers and non-quoted qualified identifiers, but
not
quoted and qualified ones. For instance:

$ echo 'include table nsp.tbl' | pg_dump --filter - >/dev/null
$echo $?
0

$ echo 'include table "TBL"' | pg_dump --filter - >/dev/null
$echo $?
0

$ echo 'include table "NSP"."TBL"' | pg_dump --filter - >/dev/null
pg_dump: error: invalid format of filter on line 1: unexpected extra data
after pattern

fixed

This should also be covered in the regression tests.

done

I'm wondering if psql's parse_identifier() could be exported and reused
here
rather than creating yet another version.

I looked there, and I don't think this parser is usable for this purpose.
It is very sensitive on white spaces, and doesn't support multi-lines. It
is designed for support readline tab complete, it is designed for
simplicity not for correctness.

Nitpicking: the comments needs some improvements:

+ /*
+  * Simple routines - just don't repeat same code
+  *
+  * Returns true, when filter's file is opened
+  */
+ bool
+ filter_init(FilterStateData *fstate, const char *filename)

done

also, is there any reason why this function doesn't call exit_nicely in
case of
error rather than letting each caller do it without any other cleanup?

It is commented few lines up

/*
* Following routines are called from pg_dump, pg_dumpall and pg_restore.
* Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
* is different from implementation of this rutine in pg_dumpall. So instead
* direct calling exit_nicely we have to return some error flag (in this
* case NULL), and exit_nicelly will be executed from caller's routine.
*/

+ /*
+  * Release allocated sources for filter
+  */
+ void
+ filter_free_sources(FilterStateData *fstate)

I'm assuming "ressources" not "sources"?

changed

+ /*
+  * log_format_error - Emit error message
+  *
+  * This is mostly a convenience routine to avoid duplicating file
closing code
+  * in multiple callsites.
+  */
+ void
+ log_invalid_filter_format(FilterStateData *fstate, char *message)

mismatch between comment and function name (same for filter_read_item)

fixes

+ static const char *
+ filter_object_type_name(FilterObjectType fot)

No description.

fixed

/*
* Helper routine to reduce duplicated code
*/
void
log_unsupported_filter_object_type(FilterStateData *fstate,

const char *appname,

FilterObjectType fot)

Need more helpful comment.

fixed

Thank you for comments

attached updated patch

Regards

Pavel

Attachments:

pg_dump-filter-20221026.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221026.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..955bfcfdad 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..67e1874224
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,467 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"
+#include "common/fe_memutils.h"
+#include "common/string.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == bytes) && (pg_strncasecmp(cstr, str, bytes) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this rutine in pg_dumpall. So instead
+ * direct calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for filter
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_DATA:
+			return "data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	return "unknown object type";
+}
+
+/*
+ * Emit error message "invalid format of filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format of filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format of filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "The application \"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern. Returns pointer to the
+ * first character after the pattern. Returns NULL on error.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	/*
+	 * If the object name pattern has been quoted, we must take care to parse
+	 * out the entire quoted pattern, which may contain whitespace and can span
+	 * many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						fstate->is_error = true;
+					}
+					else
+						log_invalid_filter_format(fstate, "unexpected end of file");
+
+					return NULL;
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * Where command is "include" or "exclude", and object_type is one of:
+ * "table", "schema", "foreign_data", "data", "database", "function",
+ * "trigger" or "index". The pattern is either simple without any
+ * whitespace, or properly quoted in case there may be whitespace in the
+ * object name. The pattern handling follows the same rules as other object
+ * include and exclude functions; it can use wildcards. Returns true, when
+ * one filter item was successfully read and parsed.  When object name
+ * contains \n chars, then more than one line from input file can be
+ * processed. Returns false when the filter file reaches EOF. In case of
+ * errors, the function wont return but will exit with an appropriate error
+ * message.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			str = filter_get_pattern(fstate, str, objname);
+			if (!str)
+				return false;
+
+			/* skip whitespaces */
+			while (isspace(*str))
+				str++;
+
+			/*
+			 * qualified identifier can be valid pattern. So repeat
+			 * reading if char after pattern is dot.
+			 */
+			if (*str == '.')
+			{
+				PQExpBuffer qual_name = createPQExpBuffer();
+
+				appendPQExpBufferStr(qual_name, *objname);
+				free(*objname);
+
+				do
+				{
+					str = filter_get_pattern(fstate, ++str, objname);
+					if (!str)
+						return false;
+
+					appendPQExpBufferChar(qual_name, '.');
+					appendPQExpBufferStr(qual_name, *objname);
+					free(*objname);
+
+					while (isspace(*str))
+						str++;
+				}
+				while (*str == '.');
+
+				*objname = qual_name->data;
+			}
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			if (*str != '\0' && *str != '#')
+			{
+				log_invalid_filter_format(fstate,
+										  "unexpected extra data after pattern");
+				return false;
+			}
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..0e17a71665
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bd9b066e4e..276be070ab 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "include filter is not allowed for this type of object");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39..e060bcc2fe 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1898,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1921,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "include database filter is not suppported");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..96ffc2455f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, optarg);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "exclude filter is not allowed for this type of object");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..e89ebaf1b9
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,514 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 69;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/The application "pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted tables are not restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#162Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#161)
Re: proposal: possibility to read dumped table's name from file

Hi,

On Wed, Oct 26, 2022 at 06:26:26AM +0200, Pavel Stehule wrote:

�t 18. 10. 2022 v 11:33 odes�latel Julien Rouhaud <rjuju123@gmail.com>
napsal:

I'm wondering if psql's parse_identifier() could be exported and reused
here
rather than creating yet another version.

I looked there, and I don't think this parser is usable for this purpose.
It is very sensitive on white spaces, and doesn't support multi-lines. It
is designed for support readline tab complete, it is designed for
simplicity not for correctness.

Ah, sorry I should have checked more thoroughly. I guess it's ok to have
another identifier parser for the include file then, as this new one wouldn't
really fit the tab-completion use case.

also, is there any reason why this function doesn't call exit_nicely in
case of
error rather than letting each caller do it without any other cleanup?

It is commented few lines up

/*
* Following routines are called from pg_dump, pg_dumpall and pg_restore.
* Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
* is different from implementation of this rutine in pg_dumpall. So instead
* direct calling exit_nicely we have to return some error flag (in this
* case NULL), and exit_nicelly will be executed from caller's routine.
*/

Oh right, I totally missed it sorry about that!

About the new version, I didn't find any problem with the feature itself so
it's a good thing!

I still have a few comments about the patch. First, about the behavior:

- is that ok to have just "data" pattern instead of "table_data" or something
like that, since it's supposed to match --exclude-table-data option?

- the error message are sometimes not super helpful. For instance:

$ echo "include data t1" | pg_dump --filter -
pg_dump: error: invalid format of filter on line 1: include filter is not allowed for this type of object

It would be nice if the error message mentioned "data" rather than a generic
"this type of object". Also, maybe we should quote "include" to outline that
we found this keyword?

About the patch itself:
filter.c:

+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"

the filter.h inclusion should be done with the rest of the includes, in
alphabetical order.

+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == bytes) && (pg_strncasecmp(cstr, str, bytes) == 0))

nit: our guidline is to protect macro arguments with parenthesis. Some
arguments can be evaluated multiple times but I don't think it's worth adding a
comment for that.

+ * Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this rutine in pg_dumpall. So instead
+ * direct calling exit_nicely we have to return some error flag (in this

typos: s/Unfortunatelly/Unfortunately/ and s/rutine/routine/
Also, it would probably be better to say "instead of directly calling..."

+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+[...]
+	}
+
+	return "unknown object type";
+}

I'm wondering if we should add a pg_unreachable() there, some compilers might
complain otherwise. See CreateDestReceiver() for instance for similar pattern.

+ * Emit error message "invalid format of filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)

nit: invalid format *in* filter file...?

+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "The application \"%s\" doesn't support filter for object type \"%s\".",

nit: there shouldn't be uppercase in error messages, especially since this will
be appended to another message by log_invalid_filter_format. I would just just
drop "The application" entirely for brevity.

+/*
+ * Release allocated resources for filter
+ */
+void
+filter_free(FilterStateData *fstate)

nit: Release allocated resources for *the given* filter?

+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL, when the buffer is empty or first
+ * char is not alpha. The length of the found keyword is returned in the size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+   [...]
+	if (isascii(*ptr) && isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+			ptr++;

Is there any reason to test isascii()? isalpha() should already cover that and
should be cheaper to test anyway.

Also nit: "Returns NULL when the buffer..." (unnecessary comma), and the '_'
char is also allowed.

+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+[...]
+			keyword = filter_get_keyword((const char **) &str, &size);

Is there any interest with the initial pg_strip_crlf? AFAICT all the rest of
the code will ignore such caracters using isspace() so it wouldn't change
anything.

Dropping both pg_strip_crlf() would allow you to declare str as const rather
than doing it in function calls. It would require to add const qualifiers in a
few other places, but it seems like an improvement, as for instance right now
filter_get_pattern is free to rewrite the str (because it's also calling
pg_strip_crlf, but there's no guarantee that it doesn't do anything else).

+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern. Returns pointer to the
+ * first character after the pattern. Returns NULL on error.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,

nit: suggestion to reword the comment, maybe something like

/*
* filter_get_pattern - Identify an object identifier pattern
*
* Try to parse an object identifier pattern from the passed buffer. If one is
* found, it sets objname to a string with the object identifier pattern and
* returns a pointer to the first byte after the found pattern. Otherwise NULL
* is returned.
*/

+bool
+filter_read_item(FilterStateData *fstate,

Another suggestion for comment rewrite:

/*-------------------
* filter_read_item - Read command/type/pattern triplet from a filter file
*
* This will parse one filter item from the filter file, and while it is a
* row based format a pattern may span more than one line due to how object
* names can be constructed. The expected format of the filter file is:
*
* <command> <object_type> <pattern>
*
* command can be "include" or "exclude"
* object_type can one of: "table", "schema", "foreign_data", "data",
* "database", "function", "trigger" or "index"
* pattern can be any possibly-quoted and possibly-qualified identifier. It
* follows the same rules as other object include and exclude functions so it
* can also use wildcards.
*
* Returns true when one filter item was successfully read and parsed. When
* object name contains \n chars, then more than one line from input file can
* be processed. Returns false when the filter file reaches EOF. In case of
* error, the function will emit an appropriate error message before returning
* false.
*/

Note also that your original comment said:
+ * In case of
+ * errors, the function wont return but will exit with an appropriate error
+ * message.

But AFAICS that's not the case: it will indeed log an appropriate error message
but will return false. I'm assuming that the comment was outdated as the
calling code handles it just fine, so I just modified the comment.

filter.h:

+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"

It's definitely not ok to include .ch in frontend code. But AFAICS just
removing it doesn't cause any problem. Note also that there should be an empty
line after the #define FILTER_H per usual coding style.

#163Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#162)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

čt 3. 11. 2022 v 5:09 odesílatel Julien Rouhaud <rjuju123@gmail.com> napsal:

Hi,

On Wed, Oct 26, 2022 at 06:26:26AM +0200, Pavel Stehule wrote:

út 18. 10. 2022 v 11:33 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

I'm wondering if psql's parse_identifier() could be exported and reused
here
rather than creating yet another version.

I looked there, and I don't think this parser is usable for this purpose.
It is very sensitive on white spaces, and doesn't support multi-lines.

It

is designed for support readline tab complete, it is designed for
simplicity not for correctness.

Ah, sorry I should have checked more thoroughly. I guess it's ok to have
another identifier parser for the include file then, as this new one
wouldn't
really fit the tab-completion use case.

also, is there any reason why this function doesn't call exit_nicely in
case of
error rather than letting each caller do it without any other cleanup?

It is commented few lines up

/*
* Following routines are called from pg_dump, pg_dumpall and pg_restore.
* Unfortunatelly, implementation of exit_nicely in pg_dump and

pg_restore

* is different from implementation of this rutine in pg_dumpall. So

instead

* direct calling exit_nicely we have to return some error flag (in this
* case NULL), and exit_nicelly will be executed from caller's routine.
*/

Oh right, I totally missed it sorry about that!

About the new version, I didn't find any problem with the feature itself so
it's a good thing!

I still have a few comments about the patch. First, about the behavior:

- is that ok to have just "data" pattern instead of "table_data" or
something
like that, since it's supposed to match --exclude-table-data option?

done

- the error message are sometimes not super helpful. For instance:

$ echo "include data t1" | pg_dump --filter -
pg_dump: error: invalid format of filter on line 1: include filter is not
allowed for this type of object

It would be nice if the error message mentioned "data" rather than a
generic
"this type of object". Also, maybe we should quote "include" to outline
that
we found this keyword?

done

About the patch itself:
filter.c:

+#include "postgres_fe.h"
+
+#include "filter.h"
+
+#include "common/logging.h"

the filter.h inclusion should be done with the rest of the includes, in
alphabetical order.

done

+#define                is_keyword_str(cstr, str, bytes) \
+       ((strlen(cstr) == bytes) && (pg_strncasecmp(cstr, str, bytes) ==
0))

nit: our guidline is to protect macro arguments with parenthesis. Some
arguments can be evaluated multiple times but I don't think it's worth
adding a
comment for that.

done

+ * Unfortunatelly, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this rutine in pg_dumpall. So
instead
+ * direct calling exit_nicely we have to return some error flag (in this

typos: s/Unfortunatelly/Unfortunately/ and s/rutine/routine/
Also, it would probably be better to say "instead of directly calling..."

done

+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+       switch (fot)
+       {
+               case FILTER_OBJECT_TYPE_NONE:
+                       return "comment or empty line";
+[...]
+       }
+
+       return "unknown object type";
+}

I'm wondering if we should add a pg_unreachable() there, some compilers
might
complain otherwise. See CreateDestReceiver() for instance for similar
pattern.

done

+ * Emit error message "invalid format of filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing
code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)

nit: invalid format *in* filter file...?

changed

+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+
const char *appname,
+
FilterObjectType fot)
+{
+       PQExpBuffer str = createPQExpBuffer();
+
+       printfPQExpBuffer(str,
+                                         "The application \"%s\" doesn't
support filter for object type \"%s\".",

nit: there shouldn't be uppercase in error messages, especially since this
will
be appended to another message by log_invalid_filter_format. I would just
just
drop "The application" entirely for brevity.

changed

+/*
+ * Release allocated resources for filter
+ */
+void
+filter_free(FilterStateData *fstate)

nit: Release allocated resources for *the given* filter?

changed

+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL, when the buffer is empty or
first
+ * char is not alpha. The length of the found keyword is returned in the
size
+ * parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+   [...]
+       if (isascii(*ptr) && isalpha(*ptr))
+       {
+               result = ptr++;
+
+               while (isascii(*ptr) && (isalpha(*ptr) || *ptr == '_'))
+                       ptr++;

Is there any reason to test isascii()? isalpha() should already cover
that and
should be cheaper to test anyway.

changed. I wanted to limit keyword's char just for basic ascii alphabets,
but the benefit probably is not too strong, and the real effect can be
messy, so I removed isascii test

Also nit: "Returns NULL when the buffer..." (unnecessary comma), and the
'_'
char is also allowed.

done

+filter_read_item(FilterStateData *fstate,
+                                bool *is_include,
+                                char **objname,
+                                FilterObjectType *objtype)
+{
+       Assert(!fstate->is_error);
+
+       if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+       {
+               char       *str = fstate->linebuff.data;
+               const char *keyword;
+               int                     size;
+
+               fstate->lineno++;
+
+               (void) pg_strip_crlf(str);
+
+               /* Skip initial white spaces */
+               while (isspace(*str))
+                       str++;
+[...]
+                       keyword = filter_get_keyword((const char **) &str,
&size);

Is there any interest with the initial pg_strip_crlf? AFAICT all the rest
of
the code will ignore such caracters using isspace() so it wouldn't change
anything.

I think reading multiline identifiers is a little bit easier, because I
don't need to check the ending \n and \r
When I read multiline identifiers, I cannot ignore white spaces.

Dropping both pg_strip_crlf() would allow you to declare str as const
rather
than doing it in function calls. It would require to add const qualifiers
in a
few other places, but it seems like an improvement, as for instance right
now
filter_get_pattern is free to rewrite the str (because it's also calling
pg_strip_crlf, but there's no guarantee that it doesn't do anything else).

+/*
+ * filter_get_pattern - Read an object identifier pattern from the buffer
+ *
+ * Parses an object identifier pattern from the passed in buffer and sets
+ * objname to a string with object identifier pattern. Returns pointer to
the
+ * first character after the pattern. Returns NULL on error.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,

nit: suggestion to reword the comment, maybe something like

/*
* filter_get_pattern - Identify an object identifier pattern
*
* Try to parse an object identifier pattern from the passed buffer. If
one is
* found, it sets objname to a string with the object identifier pattern
and
* returns a pointer to the first byte after the found pattern. Otherwise
NULL
* is returned.
*/

replaced

+bool
+filter_read_item(FilterStateData *fstate,

Another suggestion for comment rewrite:

/*-------------------
* filter_read_item - Read command/type/pattern triplet from a filter file
*
* This will parse one filter item from the filter file, and while it is a
* row based format a pattern may span more than one line due to how object
* names can be constructed. The expected format of the filter file is:
*
* <command> <object_type> <pattern>
*
* command can be "include" or "exclude"
* object_type can one of: "table", "schema", "foreign_data", "data",
* "database", "function", "trigger" or "index"
* pattern can be any possibly-quoted and possibly-qualified identifier.
It
* follows the same rules as other object include and exclude functions so
it
* can also use wildcards.
*
* Returns true when one filter item was successfully read and parsed.
When
* object name contains \n chars, then more than one line from input file
can
* be processed. Returns false when the filter file reaches EOF. In case
of
* error, the function will emit an appropriate error message before
returning
* false.
*/

replaced, thank you for the text

Note also that your original comment said:
+ * In case of
+ * errors, the function wont return but will exit with an appropriate
error
+ * message.

But AFAICS that's not the case: it will indeed log an appropriate error
message
but will return false. I'm assuming that the comment was outdated as the
calling code handles it just fine, so I just modified the comment.

yes

filter.h:

+#ifndef FILTER_H
+#define FILTER_H
+#include "c.h"

It's definitely not ok to include .ch in frontend code. But AFAICS just
removing it doesn't cause any problem. Note also that there should be an
empty
line after the #define FILTER_H per usual coding style.

fixed - it looks so it was some garbage

updated patch attached

big thanks for these comments and tips

Regards

Pavel

Attachments:

pg_dump-filter-20221103.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221103.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..d5a6e2c7ee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | table_data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..76e785d2f8
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,469 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this routine in pg_dumpall. So instead
+ * of directly calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * filter_get_pattern - Identify an object identifier pattern
+ *
+ * Try to parse an object identifier pattern from the passed buffer. If one is
+ * found, it sets objname to a string with the object identifier pattern and
+ * returns a pointer to the first byte after the found pattern. Otherwise NULL
+ * is returned.
+ */
+static char *
+filter_get_pattern(FilterStateData *fstate,
+				   char *str,
+				   char **objname)
+{
+	/* Skip whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	/*
+	 * If the object name pattern has been quoted, we must take care to parse
+	 * out the entire quoted pattern, which may contain whitespace and can span
+	 * many lines.
+	 */
+	if (*str == '"')
+	{
+		PQExpBuffer quoted_name = createPQExpBuffer();
+
+		appendPQExpBufferChar(quoted_name, '"');
+		str++;
+
+		while (1)
+		{
+			if (*str == '\0')
+			{
+				Assert(fstate->linebuff.data);
+
+				if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+				{
+					if (ferror(fstate->fp))
+					{
+						pg_log_error("could not read from filter file \"%s\": %m",
+									 fstate->filename);
+						fstate->is_error = true;
+					}
+					else
+						log_invalid_filter_format(fstate, "unexpected end of file");
+
+					return NULL;
+				}
+
+				str = fstate->linebuff.data;
+				(void) pg_strip_crlf(str);
+
+				appendPQExpBufferChar(quoted_name, '\n');
+				fstate->lineno++;
+			}
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(quoted_name, '"');
+				str++;
+
+				if (*str == '"')
+				{
+					appendPQExpBufferChar(quoted_name, '"');
+					str++;
+				}
+				else
+					break;
+			}
+			else if (*str == '\\')
+			{
+				str++;
+				if (*str == 'n')
+					appendPQExpBufferChar(quoted_name, '\n');
+				else if (*str == '\\')
+					appendPQExpBufferChar(quoted_name, '\\');
+
+				str++;
+			}
+			else
+				appendPQExpBufferChar(quoted_name, *str++);
+		}
+
+		*objname = pg_strdup(quoted_name->data);
+		destroyPQExpBuffer(quoted_name);
+	}
+	else
+	{
+		char	   *startptr = str++;
+
+		/* Simple variant, read to EOL or to first whitespace */
+		while (*str && !isspace(*str))
+			str++;
+
+		*objname = pnstrdup(startptr, str - startptr);
+	}
+
+	return str;
+}
+
+/*-------------------
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		char	   *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+
+		fstate->lineno++;
+
+		(void) pg_strip_crlf(str);
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			str = filter_get_pattern(fstate, str, objname);
+			if (!str)
+				return false;
+
+			/* skip whitespaces */
+			while (isspace(*str))
+				str++;
+
+			/*
+			 * qualified identifier can be valid pattern. So repeat
+			 * reading if char after pattern is dot.
+			 */
+			if (*str == '.')
+			{
+				PQExpBuffer qual_name = createPQExpBuffer();
+
+				appendPQExpBufferStr(qual_name, *objname);
+				free(*objname);
+
+				do
+				{
+					str = filter_get_pattern(fstate, ++str, objname);
+					if (!str)
+						return false;
+
+					appendPQExpBufferChar(qual_name, '.');
+					appendPQExpBufferStr(qual_name, *objname);
+					free(*objname);
+
+					while (isspace(*str))
+						str++;
+				}
+				while (*str == '.');
+
+				*objname = qual_name->data;
+			}
+
+			/*
+			 * Look for any content after the object identifier. Comments and
+			 * whitespace are allowed, other content may indicate that the
+			 * user needed to quote the object name so exit with an invalid
+			 * format error.
+			 */
+			if (*str != '\0' && *str != '#')
+			{
+				log_invalid_filter_format(fstate,
+										  "unexpected extra data after pattern");
+				return false;
+			}
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..251763ad8e
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index da427f4d4a..a231dc2bb3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39..4597ac3822 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1898,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1921,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..9a5cdfd6eb 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, optarg);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..e72d338372
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,514 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 69;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unexpected extra data/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted tables are not restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#164Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#163)
Re: proposal: possibility to read dumped table's name from file

On Thu, Nov 03, 2022 at 10:22:15PM +0100, Pavel Stehule wrote:

čt 3. 11. 2022 v 5:09 odesílatel Julien Rouhaud <rjuju123@gmail.com> napsal:

Is there any interest with the initial pg_strip_crlf? AFAICT all the rest
of
the code will ignore such caracters using isspace() so it wouldn't change
anything.

I think reading multiline identifiers is a little bit easier, because I
don't need to check the ending \n and \r
When I read multiline identifiers, I cannot ignore white spaces.

Ok. I don't have a strong objection to it.

updated patch attached

big thanks for these comments and tips

Thanks for the updated patch! As far as I'm concerned the patch is in a good
shape, passes the CI and I don't have anything more to say so I'm marking it as
Ready for Committer!

#165Justin Pryzby
pryzby@telsasoft.com
In reply to: Julien Rouhaud (#164)
Re: proposal: possibility to read dumped table's name from file

On Fri, Nov 04, 2022 at 07:59:01PM +0800, Julien Rouhaud wrote:

On Thu, Nov 03, 2022 at 10:22:15PM +0100, Pavel Stehule wrote:

čt 3. 11. 2022 v 5:09 odesílatel Julien Rouhaud <rjuju123@gmail.com> napsal:
updated patch attached

big thanks for these comments and tips

Thanks for the updated patch! As far as I'm concerned the patch is in a good
shape, passes the CI and I don't have anything more to say so I'm marking it as
Ready for Committer!

+1

I started looking to see if it's possible to simplify the patch at all,
but nothing to show yet.

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

--
Justin

#166Julien Rouhaud
rjuju123@gmail.com
In reply to: Justin Pryzby (#165)
Re: proposal: possibility to read dumped table's name from file

On Fri, Nov 04, 2022 at 07:19:27AM -0500, Justin Pryzby wrote:

On Fri, Nov 04, 2022 at 07:59:01PM +0800, Julien Rouhaud wrote:

On Thu, Nov 03, 2022 at 10:22:15PM +0100, Pavel Stehule wrote:

čt 3. 11. 2022 v 5:09 odesílatel Julien Rouhaud <rjuju123@gmail.com> napsal:
updated patch attached

big thanks for these comments and tips

Thanks for the updated patch! As far as I'm concerned the patch is in a good
shape, passes the CI and I don't have anything more to say so I'm marking it as
Ready for Committer!

+1

I started looking to see if it's possible to simplify the patch at all,
but nothing to show yet.

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

Ah indeed, good catch! Maybe there should be an explicit test for every
(include|exclude) / objtype combination? It would be a bit verbose (and
possibly hard to maintain).

#167Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#166)
Re: proposal: possibility to read dumped table's name from file

pá 4. 11. 2022 v 14:28 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

On Fri, Nov 04, 2022 at 07:19:27AM -0500, Justin Pryzby wrote:

On Fri, Nov 04, 2022 at 07:59:01PM +0800, Julien Rouhaud wrote:

On Thu, Nov 03, 2022 at 10:22:15PM +0100, Pavel Stehule wrote:

čt 3. 11. 2022 v 5:09 odesílatel Julien Rouhaud <rjuju123@gmail.com>

napsal:

updated patch attached

big thanks for these comments and tips

Thanks for the updated patch! As far as I'm concerned the patch is in

a good

shape, passes the CI and I don't have anything more to say so I'm

marking it as

Ready for Committer!

+1

I started looking to see if it's possible to simplify the patch at all,
but nothing to show yet.

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

Ah indeed, good catch! Maybe there should be an explicit test for every
(include|exclude) / objtype combination? It would be a bit verbose (and
possibly hard to maintain).

I'll do it

#168Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#167)
Re: proposal: possibility to read dumped table's name from file

On Fri, Nov 04, 2022 at 02:37:05PM +0100, Pavel Stehule wrote:

p� 4. 11. 2022 v 14:28 odes�latel Julien Rouhaud <rjuju123@gmail.com>
napsal:

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

Ah indeed, good catch! Maybe there should be an explicit test for every
(include|exclude) / objtype combination? It would be a bit verbose (and
possibly hard to maintain).

I'll do it

Thanks a lot Pavel! I switched the CF entry back to "Waiting on Author" in the
meantime.

#169Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#166)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

pá 4. 11. 2022 v 14:28 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

On Fri, Nov 04, 2022 at 07:19:27AM -0500, Justin Pryzby wrote:

On Fri, Nov 04, 2022 at 07:59:01PM +0800, Julien Rouhaud wrote:

On Thu, Nov 03, 2022 at 10:22:15PM +0100, Pavel Stehule wrote:

čt 3. 11. 2022 v 5:09 odesílatel Julien Rouhaud <rjuju123@gmail.com>

napsal:

updated patch attached

big thanks for these comments and tips

Thanks for the updated patch! As far as I'm concerned the patch is in

a good

shape, passes the CI and I don't have anything more to say so I'm

marking it as

Ready for Committer!

+1

I started looking to see if it's possible to simplify the patch at all,
but nothing to show yet.

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

Ah indeed, good catch! Maybe there should be an explicit test for every
(include|exclude) / objtype combination? It would be a bit verbose (and
possibly hard to maintain).

yes - pg_restore is not well covered by tests, fixed

I found another issue. The pg_restore requires a full signature of the
function and it is pretty sensitive on white spaces (pg_restore). I made a
mistake when I partially parsed patterns like SQL identifiers. It can work
for simple cases, but when I parse the function's signature it stops
working. So I rewrote the parsing pattern part. Now, I just read an input
string and I try to reduce spaces. Still multiline identifiers are
supported. Against the previous method of pattern parsing, I needed to
change just one regress test - now I am not able to detect garbage after
pattern :-/. It is possible to enter types like "double precision" or
"timestamp with time zone", without needing to check it on the server side.

When I wroted regress tests I found some issues of pg_restore filtering
options (not related to this patch)

* function's filtering doesn't support schema - when the name of function
is specified with schema, then the function is not found

* the function has to be specified with an argument type list - the
separator has to be exactly ", " string. Without space or with one space
more, the filtering doesn't work (new implementation of pattern parsing
reduces white spaces sensitivity). This is not a bug, but it is not well
documented.

* the trigger filtering is probably broken (on pg_restore side). The name
should be entered in form "tablename triggername"

attached updated patch

Regards

Pavel

Attachments:

pg_dump-filter-20221005.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221005.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..d5a6e2c7ee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | table_data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..65ffb1f432
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,480 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this routine in pg_dumpall. So instead
+ * of directly calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads an pattern from input. The pattern can be mix of
+ * single line or multi line subpatterns. Single line subpattern starts first
+ * non white space char, and ending last non space char on line or by char
+ * '#'. The white spaces inside are removed (around char ".()"), or reformated
+ * around char ',' or reduced (the multiple spaces are replaced by one).
+ * Multiline subpattern starts by double quote and ending by this char too.
+ * The escape rules are same like for SQL quoted literal.
+ *
+ * Routine signalizes error by returning NULL. Otherwise returns pointer
+ * to next char after last processed char in input string.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBuffer pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword((const char **) &str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			pattern = createPQExpBuffer();
+
+			str = read_pattern(fstate, str, pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern->data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..251763ad8e
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index da427f4d4a..a231dc2bb3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39..4597ac3822 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1898,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1921,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1ebf573ac4 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a1b06845a3
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,643 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 89;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#170Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#169)
Re: proposal: possibility to read dumped table's name from file

Hi,

On Sat, Nov 05, 2022 at 08:54:57PM +0100, Pavel Stehule wrote:

p� 4. 11. 2022 v 14:28 odes�latel Julien Rouhaud <rjuju123@gmail.com>
napsal:

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

Ah indeed, good catch! Maybe there should be an explicit test for every
(include|exclude) / objtype combination? It would be a bit verbose (and
possibly hard to maintain).

yes - pg_restore is not well covered by tests, fixed

I found another issue. The pg_restore requires a full signature of the
function and it is pretty sensitive on white spaces (pg_restore).

Argh, indeed. It's a good thing to have expanded the regression tests :)

I made a
mistake when I partially parsed patterns like SQL identifiers. It can work
for simple cases, but when I parse the function's signature it stops
working. So I rewrote the parsing pattern part. Now, I just read an input
string and I try to reduce spaces. Still multiline identifiers are
supported. Against the previous method of pattern parsing, I needed to
change just one regress test - now I am not able to detect garbage after
pattern :-/.

I'm not sure it's really problematic. It looks POLA-violation compatible with
regular pg_dump options, for instance:

$ echo "include table t1()" | pg_dump --filter - | ag CREATE
CREATE TABLE public.t1 (

$ pg_dump -t "t1()" | ag CREATE
CREATE TABLE public.t1 (

$ echo "include table t1()blabla" | pg_dump --filter - | ag CREATE
pg_dump: error: no matching tables were found

$ pg_dump -t "t1()blabla" | ag CREATE
pg_dump: error: no matching tables were found

I don't think the file parsing code should try to be smart about checking the
found patterns.

* function's filtering doesn't support schema - when the name of function
is specified with schema, then the function is not found

Ah I didn't know that. Indeed it only expect a non-qualified identifier, and
would restore any function that matches the name (and arguments), possibly
multiple ones if there are variants in different schema. That's unrelated to
this patch though.

* the function has to be specified with an argument type list - the
separator has to be exactly ", " string. Without space or with one space
more, the filtering doesn't work (new implementation of pattern parsing
reduces white spaces sensitivity). This is not a bug, but it is not well
documented.

Agreed.

attached updated patch

It looks overall good to me! I just have a few minor nitpicking complaints:

- you removed the pg_strip_clrf() calls and declared everything as "const char
*", so there's no need to explicitly cast the filter_get_keyword() arguments
anymore

Note also that the code now relies on the fact that there are some non-zero
bytes after a pattern to know that no errors happened. It's not a problem as
you should find an EOF marker anyway if CLRF were stripped.

+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this routine in pg_dumpall. So instead
+ * of directly calling exit_nicely we have to return some error flag (in this
+ * case NULL), and exit_nicelly will be executed from caller's routine.

Slight improvement:
[...]
Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
different from the one in pg_dumpall, so instead of...

+ * read_pattern - reads an pattern from input. The pattern can be mix of
+ * single line or multi line subpatterns. Single line subpattern starts first
+ * non white space char, and ending last non space char on line or by char
+ * '#'. The white spaces inside are removed (around char ".()"), or reformated
+ * around char ',' or reduced (the multiple spaces are replaced by one).
+ * Multiline subpattern starts by double quote and ending by this char too.
+ * The escape rules are same like for SQL quoted literal.
+ *
+ * Routine signalizes error by returning NULL. Otherwise returns pointer
+ * to next char after last processed char in input string.

typo: reads "a" pattern from input...

I don't fully understand the part about subpatterns, but is that necessary to
describe it? Simply saying that any valid and possibly-quoted identifier can
be parsed should make it clear that identifiers containing \n characters should
work too. Maybe also just mention that whitespaces are removed and special
care is taken to output routines in exactly the same way calling code will
expect it (that is comma-and-single-space type delimiter).

#171Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#170)
Re: proposal: possibility to read dumped table's name from file

pá 11. 11. 2022 v 9:11 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

Hi,

On Sat, Nov 05, 2022 at 08:54:57PM +0100, Pavel Stehule wrote:

pá 4. 11. 2022 v 14:28 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

But one thing I noticed is that "optarg" looks wrong here:

simple_string_list_append(&opts->triggerNames, optarg);

Ah indeed, good catch! Maybe there should be an explicit test for

every

(include|exclude) / objtype combination? It would be a bit verbose

(and

possibly hard to maintain).

yes - pg_restore is not well covered by tests, fixed

I found another issue. The pg_restore requires a full signature of the
function and it is pretty sensitive on white spaces (pg_restore).

Argh, indeed. It's a good thing to have expanded the regression tests :)

I made a
mistake when I partially parsed patterns like SQL identifiers. It can

work

for simple cases, but when I parse the function's signature it stops
working. So I rewrote the parsing pattern part. Now, I just read an input
string and I try to reduce spaces. Still multiline identifiers are
supported. Against the previous method of pattern parsing, I needed to
change just one regress test - now I am not able to detect garbage after
pattern :-/.

I'm not sure it's really problematic. It looks POLA-violation compatible
with
regular pg_dump options, for instance:

$ echo "include table t1()" | pg_dump --filter - | ag CREATE
CREATE TABLE public.t1 (

$ pg_dump -t "t1()" | ag CREATE
CREATE TABLE public.t1 (

$ echo "include table t1()blabla" | pg_dump --filter - | ag CREATE
pg_dump: error: no matching tables were found

$ pg_dump -t "t1()blabla" | ag CREATE
pg_dump: error: no matching tables were found

I don't think the file parsing code should try to be smart about checking
the
found patterns.

* function's filtering doesn't support schema - when the name of function
is specified with schema, then the function is not found

Ah I didn't know that. Indeed it only expect a non-qualified identifier,
and
would restore any function that matches the name (and arguments), possibly
multiple ones if there are variants in different schema. That's unrelated
to
this patch though.

* the function has to be specified with an argument type list - the
separator has to be exactly ", " string. Without space or with one space
more, the filtering doesn't work (new implementation of pattern parsing
reduces white spaces sensitivity). This is not a bug, but it is not well
documented.

Agreed.

attached updated patch

It looks overall good to me! I just have a few minor nitpicking
complaints:

- you removed the pg_strip_clrf() calls and declared everything as "const
char
*", so there's no need to explicitly cast the filter_get_keyword()
arguments
anymore

removed

Note also that the code now relies on the fact that there are some non-zero
bytes after a pattern to know that no errors happened. It's not a problem
as
you should find an EOF marker anyway if CLRF were stripped.

I am not sure if I understand this note well?

+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, implementation of exit_nicely in pg_dump and pg_restore
+ * is different from implementation of this routine in pg_dumpall. So
instead
+ * of directly calling exit_nicely we have to return some error flag (in
this
+ * case NULL), and exit_nicelly will be executed from caller's routine.

Slight improvement:
[...]
Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore
is
different from the one in pg_dumpall, so instead of...

+ * read_pattern - reads an pattern from input. The pattern can be mix of
+ * single line or multi line subpatterns. Single line subpattern starts
first
+ * non white space char, and ending last non space char on line or by char
+ * '#'. The white spaces inside are removed (around char ".()"), or
reformated
+ * around char ',' or reduced (the multiple spaces are replaced by one).
+ * Multiline subpattern starts by double quote and ending by this char
too.
+ * The escape rules are same like for SQL quoted literal.
+ *
+ * Routine signalizes error by returning NULL. Otherwise returns pointer
+ * to next char after last processed char in input string.

typo: reads "a" pattern from input...

fixed

I don't fully understand the part about subpatterns, but is that necessary
to
describe it? Simply saying that any valid and possibly-quoted identifier
can
be parsed should make it clear that identifiers containing \n characters
should
work too. Maybe also just mention that whitespaces are removed and special
care is taken to output routines in exactly the same way calling code will
expect it (that is comma-and-single-space type delimiter).

In this case I hit the limits of my English language skills.

I rewrote this comment, but it needs more care. Please, can you look at it?

#172Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#171)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

and updated patch

Regards

Pavel

Attachments:

pg_dump-filter-20221112.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221112.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..d5a6e2c7ee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | table_data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..830c9b2978
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,482 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads a pattern from input. Any valid (possibly quoted,
+ * possibly qualified) identifier (or function signature) should be parsed.
+ * An output pattern is reformatted. Initial and trailing spaces are removed.
+ * Inner spaces around chars with special meaning ",.()" are removed too.
+ * Other inner multi-spaces are replaced by single-space. The single-space is
+ * inserted after any comma char.
+ *
+ * Described reformatting is necessary, because backup filtering routines
+ * are sensitive on white spaces and strictly requires expected format.
+ *
+ * Routine signalizes error by returning NULL. Otherwise returns pointer
+ * to next char after last processed char in input string.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBuffer pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			pattern = createPQExpBuffer();
+
+			str = read_pattern(fstate, str, pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern->data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..251763ad8e
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index da427f4d4a..a231dc2bb3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39..4597ac3822 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1898,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1921,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1ebf573ac4 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a1b06845a3
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,643 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 89;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#173Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#171)
Re: proposal: possibility to read dumped table's name from file

On Sat, Nov 12, 2022 at 09:35:59PM +0100, Pavel Stehule wrote:

Thanks for the updated patch. Apart from the function comment it looks good to
me.

Justin, did you have any other comment on the patch?

I don't fully understand the part about subpatterns, but is that necessary
to
describe it? Simply saying that any valid and possibly-quoted identifier
can
be parsed should make it clear that identifiers containing \n characters
should
work too. Maybe also just mention that whitespaces are removed and special
care is taken to output routines in exactly the same way calling code will
expect it (that is comma-and-single-space type delimiter).

In this case I hit the limits of my English language skills.

I rewrote this comment, but it needs more care. Please, can you look at it?

I'm also not a native English speaker so I'm far for writing perfect comments
myself :)

Maybe something like

/*
* read_pattern - reads on object pattern from input
*
* This function will parse any valid identifier (quoted or not, qualified or
* not), which can also includes the full signature for routines.
* Note that this function takes special care to sanitize the detected
* identifier (removing extraneous whitespaces or other unnecessary
* characters). This is necessary as most backup/restore filtering functions
* only recognize identifiers if they are written exactly way as they are
* regenerated.
* Returns a pointer to next character after the found identifier, or NULL on
* error.
*/

#174Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#173)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

ne 13. 11. 2022 v 9:58 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

On Sat, Nov 12, 2022 at 09:35:59PM +0100, Pavel Stehule wrote:

Thanks for the updated patch. Apart from the function comment it looks
good to
me.

Justin, did you have any other comment on the patch?

I don't fully understand the part about subpatterns, but is that

necessary

to
describe it? Simply saying that any valid and possibly-quoted

identifier

can
be parsed should make it clear that identifiers containing \n

characters

should
work too. Maybe also just mention that whitespaces are removed and

special

care is taken to output routines in exactly the same way calling code

will

expect it (that is comma-and-single-space type delimiter).

In this case I hit the limits of my English language skills.

I rewrote this comment, but it needs more care. Please, can you look at

it?

I'm also not a native English speaker so I'm far for writing perfect
comments
myself :)

far better than mine :)

Thank you very much

updated patch attached

Regards

Pavel

Show quoted text

Maybe something like

/*
* read_pattern - reads on object pattern from input
*
* This function will parse any valid identifier (quoted or not, qualified
or
* not), which can also includes the full signature for routines.
* Note that this function takes special care to sanitize the detected
* identifier (removing extraneous whitespaces or other unnecessary
* characters). This is necessary as most backup/restore filtering
functions
* only recognize identifiers if they are written exactly way as they are
* regenerated.
* Returns a pointer to next character after the found identifier, or NULL
on
* error.
*/

Attachments:

pg_dump-filter-20221113.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221113.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..d5a6e2c7ee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | table_data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..3998312b33
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,481 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly way as they are
+ * regenerated.
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBuffer pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			pattern = createPQExpBuffer();
+
+			str = read_pattern(fstate, str, pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern->data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..251763ad8e
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index da427f4d4a..a231dc2bb3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 083012ca39..4597ac3822 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1890,7 +1898,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1914,3 +1921,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1ebf573ac4 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a1b06845a3
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,643 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 89;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#175Julien Rouhaud
rjuju123@gmail.com
In reply to: Pavel Stehule (#174)
Re: proposal: possibility to read dumped table's name from file

Hi,

On Sun, Nov 13, 2022 at 08:32:47PM +0100, Pavel Stehule wrote:

updated patch attached

Thanks!

Some enhancement could probably be done by a native english speaker, but apart
from that it looks good to me, so hearing no other complaints I'm marking the
CF entry as Ready for Committer!

#176Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#175)
Re: proposal: possibility to read dumped table's name from file

út 22. 11. 2022 v 6:26 odesílatel Julien Rouhaud <rjuju123@gmail.com>
napsal:

Hi,

On Sun, Nov 13, 2022 at 08:32:47PM +0100, Pavel Stehule wrote:

updated patch attached

Thanks!

Some enhancement could probably be done by a native english speaker, but
apart
from that it looks good to me, so hearing no other complaints I'm marking
the
CF entry as Ready for Committer!

Thank you very much for check and help

Regards

Pavel

#177Andres Freund
andres@anarazel.de
In reply to: Pavel Stehule (#174)
Re: proposal: possibility to read dumped table's name from file

Hi,

On 2022-11-13 20:32:47 +0100, Pavel Stehule wrote:

updated patch attached

It fails with address sanitizer that's now part of CI:

https://cirrus-ci.com/task/6031397744279552?logs=test_world#L2659

[06:33:11.271] # ==31965==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x619000000480 at pc 0x559f1ac40822 bp 0x7ffea83e1ad0 sp 0x7ffea83e1ac8
[06:33:11.271] # READ of size 1 at 0x619000000480 thread T0
[06:33:11.271] # #0 0x559f1ac40821 in read_pattern /tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:302
[06:33:11.271] # #1 0x559f1ac40e4d in filter_read_item /tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:459
[06:33:11.271] # #2 0x559f1abe6fa5 in read_dump_filters /tmp/cirrus-ci-build/src/bin/pg_dump/pg_dump.c:18229
[06:33:11.271] # #3 0x559f1ac2bb1b in main /tmp/cirrus-ci-build/src/bin/pg_dump/pg_dump.c:630
[06:33:11.271] # #4 0x7fd91fabfd09 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x23d09)
[06:33:11.271] # #5 0x559f1abe5d29 in _start (/tmp/cirrus-ci-build/tmp_install/usr/local/pgsql/bin/pg_dump+0x39d29)
[06:33:11.271] #
[06:33:11.271] # 0x619000000480 is located 0 bytes to the right of 1024-byte region [0x619000000080,0x619000000480)
[06:33:11.271] # allocated by thread T0 here:
[06:33:11.271] # #0 0x7fd91fe14e8f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
[06:33:11.271] # #1 0x559f1ac69f35 in pg_malloc_internal /tmp/cirrus-ci-build/src/common/fe_memutils.c:30
[06:33:11.271] # #2 0x559f1ac69f35 in palloc /tmp/cirrus-ci-build/src/common/fe_memutils.c:117
[06:33:11.271] #
[06:33:11.271] # SUMMARY: AddressSanitizer: heap-buffer-overflow /tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:302 in read_pattern

Greetings,

Andres Freund

#178Pavel Stehule
pavel.stehule@gmail.com
In reply to: Andres Freund (#177)
Re: proposal: possibility to read dumped table's name from file

út 22. 11. 2022 v 8:39 odesílatel Andres Freund <andres@anarazel.de> napsal:

Hi,

On 2022-11-13 20:32:47 +0100, Pavel Stehule wrote:

updated patch attached

It fails with address sanitizer that's now part of CI:

https://cirrus-ci.com/task/6031397744279552?logs=test_world#L2659

[06:33:11.271] # ==31965==ERROR: AddressSanitizer: heap-buffer-overflow on
address 0x619000000480 at pc 0x559f1ac40822 bp 0x7ffea83e1ad0 sp
0x7ffea83e1ac8
[06:33:11.271] # READ of size 1 at 0x619000000480 thread T0
[06:33:11.271] # #0 0x559f1ac40821 in read_pattern
/tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:302
[06:33:11.271] # #1 0x559f1ac40e4d in filter_read_item
/tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:459
[06:33:11.271] # #2 0x559f1abe6fa5 in read_dump_filters
/tmp/cirrus-ci-build/src/bin/pg_dump/pg_dump.c:18229
[06:33:11.271] # #3 0x559f1ac2bb1b in main
/tmp/cirrus-ci-build/src/bin/pg_dump/pg_dump.c:630
[06:33:11.271] # #4 0x7fd91fabfd09 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x23d09)
[06:33:11.271] # #5 0x559f1abe5d29 in _start
(/tmp/cirrus-ci-build/tmp_install/usr/local/pgsql/bin/pg_dump+0x39d29)
[06:33:11.271] #
[06:33:11.271] # 0x619000000480 is located 0 bytes to the right of
1024-byte region [0x619000000080,0x619000000480)
[06:33:11.271] # allocated by thread T0 here:
[06:33:11.271] # #0 0x7fd91fe14e8f in __interceptor_malloc
../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
[06:33:11.271] # #1 0x559f1ac69f35 in pg_malloc_internal
/tmp/cirrus-ci-build/src/common/fe_memutils.c:30
[06:33:11.271] # #2 0x559f1ac69f35 in palloc
/tmp/cirrus-ci-build/src/common/fe_memutils.c:117
[06:33:11.271] #
[06:33:11.271] # SUMMARY: AddressSanitizer: heap-buffer-overflow
/tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:302 in read_pattern

I'll check it

Show quoted text

Greetings,

Andres Freund

#179Pavel Stehule
pavel.stehule@gmail.com
In reply to: Andres Freund (#177)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

út 22. 11. 2022 v 8:39 odesílatel Andres Freund <andres@anarazel.de> napsal:

Hi,

On 2022-11-13 20:32:47 +0100, Pavel Stehule wrote:

updated patch attached

It fails with address sanitizer that's now part of CI:

https://cirrus-ci.com/task/6031397744279552?logs=test_world#L2659

[06:33:11.271] # ==31965==ERROR: AddressSanitizer: heap-buffer-overflow on
address 0x619000000480 at pc 0x559f1ac40822 bp 0x7ffea83e1ad0 sp
0x7ffea83e1ac8
[06:33:11.271] # READ of size 1 at 0x619000000480 thread T0
[06:33:11.271] # #0 0x559f1ac40821 in read_pattern
/tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:302
[06:33:11.271] # #1 0x559f1ac40e4d in filter_read_item
/tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:459
[06:33:11.271] # #2 0x559f1abe6fa5 in read_dump_filters
/tmp/cirrus-ci-build/src/bin/pg_dump/pg_dump.c:18229
[06:33:11.271] # #3 0x559f1ac2bb1b in main
/tmp/cirrus-ci-build/src/bin/pg_dump/pg_dump.c:630
[06:33:11.271] # #4 0x7fd91fabfd09 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x23d09)
[06:33:11.271] # #5 0x559f1abe5d29 in _start
(/tmp/cirrus-ci-build/tmp_install/usr/local/pgsql/bin/pg_dump+0x39d29)
[06:33:11.271] #
[06:33:11.271] # 0x619000000480 is located 0 bytes to the right of
1024-byte region [0x619000000080,0x619000000480)
[06:33:11.271] # allocated by thread T0 here:
[06:33:11.271] # #0 0x7fd91fe14e8f in __interceptor_malloc
../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
[06:33:11.271] # #1 0x559f1ac69f35 in pg_malloc_internal
/tmp/cirrus-ci-build/src/common/fe_memutils.c:30
[06:33:11.271] # #2 0x559f1ac69f35 in palloc
/tmp/cirrus-ci-build/src/common/fe_memutils.c:117
[06:33:11.271] #
[06:33:11.271] # SUMMARY: AddressSanitizer: heap-buffer-overflow
/tmp/cirrus-ci-build/src/bin/pg_dump/filter.c:302 in read_pattern

should be fixed in attached patch

I found and fix small memleak 24bytes per filter row (PQExpBufferData)

Regards

Pavel

Show quoted text

Greetings,

Andres Freund

Attachments:

pg_dump-filter-20221122.patchtext/x-patch; charset=US-ASCII; name=pg_dump-filter-20221122.patchDownload
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8b9d9f4cad..d5a6e2c7ee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -779,6 +779,80 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | schema | foreign_data | table_data } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1119,6 +1193,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) qualifier
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table qualifiers find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1528,6 +1603,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 9dc5a784dd..700e1400c8 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	$(WIN32RES) \
 	compress_io.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -43,8 +44,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..500db870c2
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,481 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly way as they are
+ * regenerated.
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..251763ad8e
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TRIGGER,
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index e66f632b54..c099d66450 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -1,6 +1,7 @@
 pg_dump_common_sources = files(
   'compress_io.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -86,6 +87,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index da427f4d4a..a231dc2bb3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -318,6 +319,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -390,6 +392,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 12},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -623,6 +626,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 12:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1028,6 +1035,8 @@ help(const char *progname)
 			 "                               access to)\n"));
 	printf(_("  --exclude-table-data=PATTERN do NOT dump data for the specified table(s)\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18198,3 +18207,89 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index a87262e333..6e99e4da5d 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1900,7 +1908,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1924,3 +1931,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1ebf573ac4 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a1b06845a3
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,643 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 89;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..d3c7292203 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
#180Gregory Stark (as CFM)
stark.cfm@gmail.com
In reply to: Julien Rouhaud (#175)
Re: proposal: possibility to read dumped table's name from file

So.... This patch has been through a lot of commitfests. And it really
doesn't seem that hard to resolve -- Pavel has seemingly been willing
to go along whichever way the wind has been blowing but honestly it
kind of seems like he's just gotten drive-by suggestions and he's put
a lot of work into trying to satisfy them.

He implemented --include-tables-from-file=... etc. Then he implemented
a hand-written parser for a DSL to select objects, then he implemented
a bison parser, then he went back to the hand-written parser.

Can we get some consensus on whether the DSL looks right and whether
the hand-written parser is sensible. And if so then can a committer
step up to actual review and commit the patch? The last review said it
might need a native English speaker to tweak some wording but
otherwise looked good.

--
Gregory Stark
As Commitfest Manager

#181Daniel Gustafsson
daniel@yesql.se
In reply to: Gregory Stark (as CFM) (#180)
Re: proposal: possibility to read dumped table's name from file

On 6 Mar 2023, at 21:45, Gregory Stark (as CFM) <stark.cfm@gmail.com> wrote:

So.... This patch has been through a lot of commitfests. And it really
doesn't seem that hard to resolve -- Pavel has seemingly been willing
to go along whichever way the wind has been blowing but honestly it
kind of seems like he's just gotten drive-by suggestions and he's put
a lot of work into trying to satisfy them.

Agreed.

He implemented --include-tables-from-file=... etc. Then he implemented
a hand-written parser for a DSL to select objects, then he implemented
a bison parser, then he went back to the hand-written parser.

Well, kind of. I was trying to take the patch to the finishing line but was
uncomfortable with the hand written parser so I implemented a parser in Bison
to replace it with. Not that hand-written parsers are bad per se (or that my
bison parser was perfect), but reading quoted identifiers across line
boundaries tend to require a fair amount of handwritten code. Pavel did not
object to this version, but it was objected to by two other committers.

At this point [0]098531E1-FBA9-4B7D-884E-0A4363EEE6DF@yesql.se I stepped down from trying to finish it as the approach I was
comfortable didn't gain traction (which is totally fine).

Downthread from this the patch got a lot of reviews from Julien with the old
parser back in place.

Can we get some consensus on whether the DSL looks right

I would consider this pretty settled.

and whether the hand-written parser is sensible.

This is the part where a committer who wants to pursue the hand-written parser
need to step up. With the amount of review received it's hopefully pretty close.

--
Daniel Gustafsson

[0]: 098531E1-FBA9-4B7D-884E-0A4363EEE6DF@yesql.se

#182Julien Rouhaud
rjuju123@gmail.com
In reply to: Daniel Gustafsson (#181)
Re: proposal: possibility to read dumped table's name from file

Hi,

On Mon, Mar 06, 2023 at 10:20:32PM +0100, Daniel Gustafsson wrote:

On 6 Mar 2023, at 21:45, Gregory Stark (as CFM) <stark.cfm@gmail.com> wrote:

So.... This patch has been through a lot of commitfests. And it really
doesn't seem that hard to resolve -- Pavel has seemingly been willing
to go along whichever way the wind has been blowing but honestly it
kind of seems like he's just gotten drive-by suggestions and he's put
a lot of work into trying to satisfy them.

Agreed.

Indeed, I'm not sure I would have had that much patience.

He implemented --include-tables-from-file=... etc. Then he implemented
a hand-written parser for a DSL to select objects, then he implemented
a bison parser, then he went back to the hand-written parser.

Well, kind of. I was trying to take the patch to the finishing line but was
uncomfortable with the hand written parser so I implemented a parser in Bison
to replace it with. Not that hand-written parsers are bad per se (or that my
bison parser was perfect), but reading quoted identifiers across line
boundaries tend to require a fair amount of handwritten code. Pavel did not
object to this version, but it was objected to by two other committers.

At this point [0] I stepped down from trying to finish it as the approach I was
comfortable didn't gain traction (which is totally fine).

Downthread from this the patch got a lot of reviews from Julien with the old
parser back in place.

Yeah, and the current state seems quite good to me.

Can we get some consensus on whether the DSL looks right

I would consider this pretty settled.

Agreed.

#183Pavel Stehule
pavel.stehule@gmail.com
In reply to: Julien Rouhaud (#182)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

út 7. 3. 2023 v 3:47 odesílatel Julien Rouhaud <rjuju123@gmail.com> napsal:

Hi,

On Mon, Mar 06, 2023 at 10:20:32PM +0100, Daniel Gustafsson wrote:

On 6 Mar 2023, at 21:45, Gregory Stark (as CFM) <stark.cfm@gmail.com>

wrote:

So.... This patch has been through a lot of commitfests. And it really
doesn't seem that hard to resolve -- Pavel has seemingly been willing
to go along whichever way the wind has been blowing but honestly it
kind of seems like he's just gotten drive-by suggestions and he's put
a lot of work into trying to satisfy them.

Agreed.

Indeed, I'm not sure I would have had that much patience.

He implemented --include-tables-from-file=... etc. Then he implemented
a hand-written parser for a DSL to select objects, then he implemented
a bison parser, then he went back to the hand-written parser.

Well, kind of. I was trying to take the patch to the finishing line but

was

uncomfortable with the hand written parser so I implemented a parser in

Bison

to replace it with. Not that hand-written parsers are bad per se (or

that my

bison parser was perfect), but reading quoted identifiers across line
boundaries tend to require a fair amount of handwritten code. Pavel did

not

object to this version, but it was objected to by two other committers.

At this point [0] I stepped down from trying to finish it as the

approach I was

comfortable didn't gain traction (which is totally fine).

Downthread from this the patch got a lot of reviews from Julien with the

old

parser back in place.

Yeah, and the current state seems quite good to me.

Can we get some consensus on whether the DSL looks right

I would consider this pretty settled.

Agreed.

rebase + enhancing about related option from a563c24

Regards

Pavel

Attachments:

0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=0001-possibility-to-read-options-for-dump-from-file.patchDownload
From 76f224be5f88b2a33f9bdc4a38a5bd49c86a7d3b Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 110 +++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 489 ++++++++++++++
 src/bin/pg_dump/filter.h                    |  56 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 117 ++++
 src/bin/pg_dump/pg_dumpall.c                |  61 +-
 src/bin/pg_dump/pg_restore.c                | 114 ++++
 src/bin/pg_dump/t/004_pg_dump_filterfile.pl | 701 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1700 insertions(+), 3 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/004_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index e6b003bf10..2a8cbe41bc 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -829,6 +829,102 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-and-children</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | table_and_children | schema | foreign_data | table_data | table_data_and_children } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>, except that
+           it also includes any partitions or inheritance child
+           tables of the table(s) matching the
+           <replaceable class="parameter">pattern</replaceable>.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1169,6 +1265,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1591,6 +1688,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e62d05e5ab..9cad26bbe6 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index eb8f59459a..bff55b6b1a 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -30,6 +30,7 @@ OBJS = \
 	compress_lz4.o \
 	compress_none.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -47,8 +48,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..971b3d93eb
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,489 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table_data_and_children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table_and_children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly way as they are
+ * regenerated.
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("table_data_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("table_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..28c5c9c834
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index ab4c25c781..57562c32f8 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -6,6 +6,7 @@ pg_dump_common_sources = files(
   'compress_lz4.c',
   'compress_none.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -96,6 +97,7 @@ tests += {
       't/001_basic.pl',
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
+      't/004_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2e068c6620..6e9475cd8a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -325,6 +326,7 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -403,6 +405,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 15},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -655,6 +658,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 15:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1101,6 +1108,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18369,3 +18378,111 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data and children filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index cd421c5944..1bfa1c8905 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1908,7 +1916,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1932,3 +1939,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1ebf573ac4 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/004_pg_dump_filterfile.pl b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..ea1702263a
--- /dev/null
+++ b/src/bin/pg_dump/t/004_pg_dump_filterfile.pl
@@ -0,0 +1,701 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 96;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !=~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index e3ffc653e5..fca7b7d0de 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.40.0

#184Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#183)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

fresh rebase

regards

Pavel

Attachments:

v20230318-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20230318-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From 09c64d9a5f2ed033c2691ca25ca6bec23320e1b0 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 110 +++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 489 ++++++++++++++
 src/bin/pg_dump/filter.h                    |  56 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 117 ++++
 src/bin/pg_dump/pg_dumpall.c                |  61 +-
 src/bin/pg_dump/pg_restore.c                | 114 ++++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 701 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1700 insertions(+), 3 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index d6b1faa804..5672fbb273 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -829,6 +829,102 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-and-children</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | table_and_children | schema | foreign_data | table_data | table_data_and_children } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>, except that
+           it also includes any partitions or inheritance child
+           tables of the table(s) matching the
+           <replaceable class="parameter">pattern</replaceable>.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1159,6 +1255,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1581,6 +1678,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858..ca2f93652c 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index eb8f59459a..bff55b6b1a 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -30,6 +30,7 @@ OBJS = \
 	compress_lz4.o \
 	compress_none.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -47,8 +48,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..971b3d93eb
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,489 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table_data_and_children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table_and_children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly way as they are
+ * regenerated.
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("table_data_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("table_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..28c5c9c834
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index b2fb7ac77f..0a626e6cc6 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -6,6 +6,7 @@ pg_dump_common_sources = files(
   'compress_lz4.c',
   'compress_none.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -97,6 +98,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d62780a088..c7cc4b1573 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -326,6 +327,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -404,6 +406,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 15},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -656,6 +659,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 15:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1102,6 +1109,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18475,3 +18484,111 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data and children filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index cd421c5944..1bfa1c8905 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1908,7 +1916,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1932,3 +1939,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1ebf573ac4 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..ea1702263a
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,701 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 96;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !=~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index e3ffc653e5..fca7b7d0de 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.40.0

#185Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#183)
3 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote:

rebase + enhancing about related option from a563c24

Thanks.

It looks like this doesn't currently handle extensions, which were added
at 6568cef26e.

+           <literal>table_and_children</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>, except that
+           it also includes any partitions or inheritance child
+           tables of the table(s) matching the
+           <replaceable class="parameter">pattern</replaceable>.

Why doesn't this just say "works like --table-and-children" ?

I think as you wrote log_invalid_filter_format(), the messages wouldn't
be translated, because they're printed via %s. One option is to call
_() on the message.

+ok($dump !=~ qr/^CREATE TABLE public\.bootab/m, "exclude dumped children table");

!=~ is being interpretted as as numeric "!=" and throwing warnings.
It should be a !~ b, right ?
It'd be nice if perl warnings during the tests were less easy to miss.

+ * char is not alpha. The char '_' is allowed too (exclude first position).

Why is it treated specially? Could it be treated the same as alpha?

+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data and children filter is not allowed");

For these, it might be better to write the literal option:

+ "include filter for \"table_data_and_children\" is not allowed");

Because the option is a literal and shouldn't be translated.
And it's probably better to write that using %s, like:

+ "include filter for \"%s\" is not allowed");

That makes shorter and fewer strings.

Find attached a bunch of other corrections as 0002.txt

I also dug up what I'd started in november, trying to reduce the code
duplication betwen pg_restore/dump/all. This isn't done, but I might
never finish it, so I'll at least show what I have in case you think
it's a good idea. This passes tests on CI, except for autoconf, due to
using exit_nicely() differently.

--
Justin

Attachments:

0001-possibility-to-read-options-for-dump-from-file.txttext/x-diff; charset=us-asciiDownload
From e86d4dca9b4754185a1d559fdea7bf4ee2de8b1a Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <okbob@github.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH 1/3] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 110 +++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 489 ++++++++++++++
 src/bin/pg_dump/filter.h                    |  56 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 117 ++++
 src/bin/pg_dump/pg_dumpall.c                |  61 +-
 src/bin/pg_dump/pg_restore.c                | 114 ++++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 701 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1700 insertions(+), 3 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index d6b1faa8042..5672fbb273b 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -829,6 +829,102 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-and-children</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | table_and_children | schema | foreign_data | table_data | table_data_and_children } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>, except that
+           it also includes any partitions or inheritance child
+           tables of the table(s) matching the
+           <replaceable class="parameter">pattern</replaceable>.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1159,6 +1255,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1581,6 +1678,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858e..ca2f93652cd 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda06..ffeb564c520 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index eb8f59459a1..bff55b6b1a2 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -30,6 +30,7 @@ OBJS = \
 	compress_lz4.o \
 	compress_none.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -47,8 +48,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 00000000000..971b3d93eb1
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,489 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table_data_and_children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table_and_children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or first
+ * char is not alpha. The char '_' is allowed too (exclude first position).
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, "unexpected end of file");
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly way as they are
+ * regenerated.
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, "missing object name pattern");
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "schema", "foreign_data", "table_data",
+ * "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   "no filter command found (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  "invalid filter command (expected \"include\" or \"exclude\")");
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, "missing filter object type");
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("table_data_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("table_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, "unsupported filter object type: \"%.*s\"", size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 00000000000..28c5c9c8346
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+
+extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+												const char *appname, FilterObjectType fot);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index b2fb7ac77fd..0a626e6cc6f 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -6,6 +6,7 @@ pg_dump_common_sources = files(
   'compress_lz4.c',
   'compress_none.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -97,6 +98,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d62780a0880..c7cc4b15736 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -326,6 +327,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -404,6 +406,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 15},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -656,6 +659,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 15:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1102,6 +1109,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18475,3 +18484,111 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"include\" table data and children filter is not allowed");
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										  "\"exclude\" foreign data filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index cd421c59443..1bfa1c89051 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1908,7 +1916,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1932,3 +1939,55 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * In this moment only excluded databases can be filtered.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"include\" database filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a1006347..1ebf573ac48 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,109 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifer patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" function filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" type  filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" table filter is not allowed");
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_invalid_filter_format(&fstate,
+										   "\"exclude\" trigger filter is not allowed");
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 00000000000..ea1702263a6
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,701 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 96;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/"exclude" foreign data filter is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !=~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !=~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 0c89c8e5297..0acd9fc89e9 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -437,6 +437,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.34.1

0002-little-fixes.txttext/x-diff; charset=us-asciiDownload
From c30d5a01d2d78ce87025e8b334e78c5d95937df3 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 17 Mar 2023 22:22:56 -0500
Subject: [PATCH 2/3] little fixes

---
 doc/src/sgml/ref/pg_dump.sgml               |  2 +-
 doc/src/sgml/ref/pg_dumpall.sgml            |  2 +-
 src/bin/pg_dump/filter.c                    | 13 +++++++------
 src/bin/pg_dump/pg_dump.c                   |  2 +-
 src/bin/pg_dump/pg_dumpall.c                |  4 ++--
 src/bin/pg_dump/pg_restore.c                |  4 ++--
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl |  8 ++++----
 7 files changed, 18 insertions(+), 17 deletions(-)

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 5672fbb273b..f7c4304aec4 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -843,7 +843,7 @@ PostgreSQL documentation
         <option>-n</option>/<option>--schema</option> for schemas,
         <option>--include-foreign-data</option> for data on foreign servers and
         <option>--exclude-table-data</option>,
-        <option>--exclude-table-data-and-and-children</option> for table data.
+        <option>--exclude-table-data-and-children</option> for table data.
         To read from <literal>STDIN</literal>, use <filename>-</filename> as the
         filename.  The <option>--filter</option> option can be specified in
         conjunction with the above listed options for including or excluding
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index ca2f93652cd..547fe3803f4 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -127,7 +127,7 @@ PostgreSQL documentation
       <listitem>
        <para>
         Specify a filename from which to read patterns for databases excluded
-        from dump. The patterns are interpretted according to the same rules
+        from the dump. The patterns are interpretted according to the same rules
         as <option>--exclude-database</option>.
         To read from <literal>STDIN</literal>, use <filename>-</filename> as the
         filename.  The <option>--filter</option> option can be specified in
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 971b3d93eb1..41fb31b7253 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -25,7 +25,7 @@
  * Following routines are called from pg_dump, pg_dumpall and pg_restore.
  * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
  * different from the one in pg_dumpall, so instead of calling exit_nicely we
- * have to return some error flag (in this case NULL), and exit_nicelly will be
+ * have to return some error flag (in this case NULL), and exit_nicely will be
  * executed from caller's routine.
  */
 
@@ -162,7 +162,7 @@ log_unsupported_filter_object_type(FilterStateData *fstate,
  *
  * Search for keywords (limited to ascii alphabetic characters) in
  * the passed in line buffer. Returns NULL when the buffer is empty or first
- * char is not alpha. The char '_' is allowed too (exclude first position).
+ * char is not alpha. The char '_' is allowed, except as the first character.
  * The length of the found keyword is returned in the size parameter.
  */
 static const char *
@@ -280,8 +280,9 @@ read_quoted_string(FilterStateData *fstate,
  * Note that this function takes special care to sanitize the detected
  * identifier (removing extraneous whitespaces or other unnecessary
  * characters).  This is necessary as most backup/restore filtering functions
- * only recognize identifiers if they are written exactly way as they are
- * regenerated.
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
  * Returns a pointer to next character after the found identifier, or NULL on
  * error.
  */
@@ -364,8 +365,8 @@ read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
  * <command> <object_type> <pattern>
  *
  * command can be "include" or "exclude"
- * object_type can one of: "table", "schema", "foreign_data", "table_data",
- * "database", "function", "trigger" or "index"
+ * object_type can one of: "table", "table_and_children", "schema", "foreign_data",
+ * "table_data", "table_and_children", "database", "function", "trigger" or "index"
  * pattern can be any possibly-quoted and possibly-qualified identifier.  It
  * follows the same rules as other object include and exclude functions so it
  * can also use wildcards.
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c7cc4b15736..42b8f8d91d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -18486,7 +18486,7 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 }
 
 /*
- * read_dump_filters - retrieve object identifer patterns from file
+ * read_dump_filters - retrieve object identifier patterns from file
  *
  * Parse the specified filter file for include and exclude patterns, and add
  * them to the relevant lists.  If the filename is "-" then filters will be
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 1bfa1c89051..d58eaaaa063 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -1941,13 +1941,13 @@ hash_string_pointer(char *s)
 }
 
 /*
- * read_dumpall_filters - retrieve database identifer patterns from file
+ * read_dumpall_filters - retrieve database identifier patterns from file
  *
  * Parse the specified filter file for include and exclude patterns, and add
  * them to the relevant lists.  If the filename is "-" then filters will be
  * read from STDIN rather than a file.
  *
- * In this moment only excluded databases can be filtered.
+ * At the moment, the only allowed filter is for database exclusion.
  */
 static void
 read_dumpall_filters(const char *filename, SimpleStringList *pattern)
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 1ebf573ac48..0ea71351737 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -504,7 +504,7 @@ usage(const char *progname)
 }
 
 /*
- * read_restore_filters - retrieve object identifer patterns from file
+ * read_restore_filters - retrieve object identifier patterns from file
  *
  * Parse the specified filter file for include and exclude patterns, and add
  * them to the relevant lists.  If the filename is "-" then filters will be
@@ -553,7 +553,7 @@ read_restore_filters(const char *filename, RestoreOptions *opts)
 			else
 			{
 				log_invalid_filter_format(&fstate,
-										   "\"exclude\" type  filter is not allowed");
+										   "\"exclude\" index filter is not allowed");
 				break;
 			}
 		}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
index ea1702263a6..67314dbd60e 100644
--- a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -640,8 +640,8 @@ command_ok(
 
 $dump = slurp_file($plainfile);
 
-ok($dump !=~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
-ok($dump !=~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
 ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
 ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
 
@@ -680,7 +680,7 @@ command_ok(
 
 $dump = slurp_file($plainfile);
 
-ok($dump !=~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
 
 open $inputfile, '>', "$tempdir/inputfile.txt"
   or die "unable to open filterfile for writing";
@@ -698,4 +698,4 @@ command_ok(
 $dump = slurp_file($plainfile);
 
 ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
-ok($dump !=~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
-- 
2.34.1

0003-f-rewrite-using-a-single-filtering-function.txttext/x-diff; charset=us-asciiDownload
From 3ea88b30bc696d7e80058e2f1ec1d2d5ad36bd7e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Thu, 3 Nov 2022 18:09:14 -0500
Subject: [PATCH 3/3] f!rewrite using a single filtering function

---
 src/bin/pg_dump/filter.c             | 173 +++++++++++++++++-
 src/bin/pg_dump/filter.h             |  71 +++++---
 src/bin/pg_dump/pg_backup.h          |  17 +-
 src/bin/pg_dump/pg_backup_archiver.c |  12 +-
 src/bin/pg_dump/pg_dump.c            | 257 +++++++--------------------
 src/bin/pg_dump/pg_dumpall.c         |  69 ++-----
 src/bin/pg_dump/pg_restore.c         | 167 +++++------------
 7 files changed, 350 insertions(+), 416 deletions(-)

diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 41fb31b7253..d52f0c7f0bf 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -15,12 +15,27 @@
 #include "common/logging.h"
 #include "common/string.h"
 #include "filter.h"
+#include "pg_backup.h"
+#include "pg_backup_utils.h"
 #include "lib/stringinfo.h"
 #include "pqexpbuffer.h"
 
 #define		is_keyword_str(cstr, str, bytes) \
 	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
 
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE		*fp;
+	const char	*filename;
+	int		lineno;
+	StringInfoData	linebuff;
+	bool		is_error;
+} FilterStateData;
+
+
 /*
  * Following routines are called from pg_dump, pg_dumpall and pg_restore.
  * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
@@ -33,7 +48,7 @@
  * Opens filter's file and initialize fstate structure.
  * Returns true on success.
  */
-bool
+static bool
 filter_init(FilterStateData *fstate, const char *filename)
 {
 	fstate->filename = filename;
@@ -60,7 +75,7 @@ filter_init(FilterStateData *fstate, const char *filename)
 /*
  * Release allocated resources for the given filter.
  */
-void
+static void
 filter_free(FilterStateData *fstate)
 {
 	free(fstate->linebuff.data);
@@ -118,7 +133,7 @@ filter_object_type_name(FilterObjectType fot)
  * This is mostly a convenience routine to avoid duplicating file closing code
  * in multiple callsites.
  */
-void
+static void
 log_invalid_filter_format(FilterStateData *fstate, char *message)
 {
 	if (fstate->fp != stdin)
@@ -136,23 +151,26 @@ log_invalid_filter_format(FilterStateData *fstate, char *message)
 	fstate->is_error = true;
 }
 
+
 /*
  * Emit error message "The application doesn't support filter for object type ..."
  *
  * This is mostly a convenience routine to avoid duplicating file closing code
  * in multiple callsites.
  */
-void
+static void
 log_unsupported_filter_object_type(FilterStateData *fstate,
 									const char *appname,
-									FilterObjectType fot)
+									FilterObjectType fot,
+									bool is_include)
 {
 	PQExpBuffer str = createPQExpBuffer();
 
 	printfPQExpBuffer(str,
-					  "\"%s\" doesn't support filter for object type \"%s\".",
+					  "\"%s\" doesn't support filter for object type \"%s\": %s",
 					  appname,
-					  filter_object_type_name(fot));
+					  filter_object_type_name(fot),
+					  is_include ? "include" : "exclude");
 
 	log_invalid_filter_format(fstate, str->data);
 }
@@ -377,7 +395,7 @@ read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
  * error, the function will emit an appropriate error message before returning
  * false.
  */
-bool
+static bool
 filter_read_item(FilterStateData *fstate,
 				 bool *is_include,
 				 char **objname,
@@ -488,3 +506,142 @@ filter_read_item(FilterStateData *fstate,
 
 	return false;
 }
+
+/*
+ * read_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+void
+read_filters(const char *appname, const char *filename, struct FilterOpts *filter_opts, struct type_opts *opts, bool *include_everything, int allow_include, int allow_exclude)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1); //
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		/* allows "table and children" whenever table is allowed */
+		if (allow_include & FILTER_OBJECT_TYPE_TABLE)
+			allow_include |= FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+		if (allow_include & FILTER_OBJECT_TYPE_TABLE_DATA)
+			allow_include |= FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+		if (allow_exclude & FILTER_OBJECT_TYPE_TABLE)
+			allow_exclude |= FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+		if (allow_exclude & FILTER_OBJECT_TYPE_TABLE_DATA)
+			allow_exclude |= FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+
+		if (is_include && (allow_include & objtype) == 0)
+			log_unsupported_filter_object_type(&fstate, appname, objtype, is_include);
+		else if (!is_include && (allow_exclude & objtype) == 0)
+			log_unsupported_filter_object_type(&fstate, appname, objtype, is_include);
+		else if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			Assert(!is_include);
+			simple_string_list_append(&filter_opts->database_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			// Assert(is_include);
+			if (!is_include)
+				log_invalid_filter_format(&fstate,
+						"\"exclude\" foreign data filter is not allowed");
+			else
+				simple_string_list_append(&filter_opts->foreign_servers_include_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			Assert(is_include);
+			opts->selTypes = 1;
+			opts->selFunction = 1;
+			simple_string_list_append(&filter_opts->function_include_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			Assert(is_include);
+			opts->selTypes = 1;
+			opts->selIndex = 1;
+			simple_string_list_append(&filter_opts->index_include_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&filter_opts->schema_include_patterns, objname);
+				*include_everything = false;
+			}
+			else
+				simple_string_list_append(&filter_opts->schema_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				*include_everything = false;
+				simple_string_list_append(&filter_opts->table_include_patterns, objname);
+			}
+			else
+				simple_string_list_append(&filter_opts->table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			Assert(!is_include);
+			simple_string_list_append(&filter_opts->tabledata_exclude_patterns,
+									  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			Assert(!is_include);
+			simple_string_list_append(&filter_opts->tabledata_exclude_patterns_and_children,
+									  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&filter_opts->table_include_patterns_and_children, objname);
+				*include_everything = false;
+			}
+			else
+				simple_string_list_append(&filter_opts->table_exclude_patterns_and_children, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			Assert(is_include);
+			opts->selTypes = 1;
+			opts->selTrigger = 1;
+			simple_string_list_append(&filter_opts->trigger_include_patterns, objname);
+		}
+		else
+		{
+			// log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			// log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			// log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			pg_fatal("unhandled filter type: %d", objtype);
+		}
+
+		if (objname)
+			free(objname);
+
+		if (fstate.is_error)
+			exit_nicely(1); //
+	}
+
+	if (fstate.is_error)
+		exit_nicely(1); //
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
index 28c5c9c8346..99a2a5a9a07 100644
--- a/src/bin/pg_dump/filter.h
+++ b/src/bin/pg_dump/filter.h
@@ -14,43 +14,56 @@
 #define FILTER_H
 
 #include "lib/stringinfo.h"
-
-/*
- * State data for reading filter items from stream
- */
-typedef struct
-{
-	FILE	   *fp;
-	const char *filename;
-	int			lineno;
-	StringInfoData linebuff;
-	bool		is_error;
-}			FilterStateData;
+#include "fe_utils/simple_list.h"
 
 /*
  * List of objects that can be specified in filter file
  */
 typedef enum
 {
-	FILTER_OBJECT_TYPE_NONE,
-	FILTER_OBJECT_TYPE_TABLE_DATA,
-	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
-	FILTER_OBJECT_TYPE_DATABASE,
-	FILTER_OBJECT_TYPE_FOREIGN_DATA,
-	FILTER_OBJECT_TYPE_FUNCTION,
-	FILTER_OBJECT_TYPE_INDEX,
-	FILTER_OBJECT_TYPE_SCHEMA,
-	FILTER_OBJECT_TYPE_TABLE,
-	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
-	FILTER_OBJECT_TYPE_TRIGGER
+	FILTER_OBJECT_TYPE_NONE = 1<<0,
+	FILTER_OBJECT_TYPE_TABLE_DATA = 1<<1,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN = 1<<2,
+	FILTER_OBJECT_TYPE_DATABASE = 1<<3,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA = 1<<4,
+	FILTER_OBJECT_TYPE_FUNCTION = 1<<5,
+	FILTER_OBJECT_TYPE_INDEX = 1<<6,
+	FILTER_OBJECT_TYPE_SCHEMA = 1<<7,
+	FILTER_OBJECT_TYPE_TABLE = 1<<8,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN = 1<<9,
+	FILTER_OBJECT_TYPE_TRIGGER = 1<<10,
 }			FilterObjectType;
 
-extern bool filter_init(FilterStateData *fstate, const char *filename);
-extern void filter_free(FilterStateData *fstate);
+struct FilterOpts {
+	SimpleStringList schema_include_patterns;
+	SimpleOidList schema_include_oids;
+	SimpleStringList schema_exclude_patterns;
+	SimpleOidList schema_exclude_oids;
+
+	SimpleStringList table_include_patterns;
+	SimpleStringList table_include_patterns_and_children;
+	SimpleOidList table_include_oids;
+	SimpleStringList table_exclude_patterns;
+	SimpleStringList table_exclude_patterns_and_children;
+	SimpleOidList table_exclude_oids;
+	SimpleStringList tabledata_exclude_patterns;
+	SimpleStringList tabledata_exclude_patterns_and_children;
+	SimpleOidList tabledata_exclude_oids;
+	SimpleStringList foreign_servers_include_patterns;
+	SimpleOidList foreign_servers_include_oids;
+
+	SimpleStringList extension_include_patterns;
+	SimpleOidList extension_include_oids;
+
+	SimpleStringList database_exclude_patterns;
+	SimpleStringList index_include_patterns;
+	SimpleStringList function_include_patterns;
+	SimpleStringList trigger_include_patterns;
+};
 
-extern void log_invalid_filter_format(FilterStateData *fstate, char *message);
-extern void log_unsupported_filter_object_type(FilterStateData *fstate,
-												const char *appname, FilterObjectType fot);
-extern bool filter_read_item(FilterStateData *fstate, bool *is_include, char **objname, FilterObjectType *objtype);
+struct type_opts; /* fwd decl */
+extern void read_filters(const char *appname, const char *filename, struct FilterOpts *filteropts,
+		struct type_opts *opts, bool *include_everything,
+		int allow_include, int allow_exclude);
 
 #endif
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index aba780ef4b1..0794ec3a301 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -26,6 +26,7 @@
 #include "common/compression.h"
 #include "fe_utils/simple_list.h"
 #include "libpq-fe.h"
+#include "filter.h"
 
 
 typedef enum trivalue
@@ -90,6 +91,15 @@ typedef struct _connParams
 	char	   *override_dbname;
 } ConnParams;
 
+struct type_opts
+{
+	int			selTypes;
+	int			selIndex;
+	int			selFunction;
+	int			selTrigger;
+	int			selTable;
+};
+
 typedef struct _restoreOptions
 {
 	int			createDB;		/* Issue commands to create the database */
@@ -127,11 +137,8 @@ typedef struct _restoreOptions
 	int			format;
 	char	   *formatName;
 
-	int			selTypes;
-	int			selIndex;
-	int			selFunction;
-	int			selTrigger;
-	int			selTable;
+	struct type_opts	typeopts;
+
 	SimpleStringList indexNames;
 	SimpleStringList functionNames;
 	SimpleStringList schemaNames;
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 3337d34e405..57e73bd608d 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -2890,7 +2890,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
 		}
 		else if (ropt->schemaNames.head != NULL ||
 				 ropt->schemaExcludeNames.head != NULL ||
-				 ropt->selTypes)
+				 ropt->typeopts.selTypes)
 		{
 			/*
 			 * In a selective dump/restore, we want to restore these dependent
@@ -2930,7 +2930,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
 			simple_string_list_member(&ropt->schemaExcludeNames, te->namespace))
 			return 0;
 
-		if (ropt->selTypes)
+		if (ropt->typeopts.selTypes)
 		{
 			if (strcmp(te->desc, "TABLE") == 0 ||
 				strcmp(te->desc, "TABLE DATA") == 0 ||
@@ -2941,7 +2941,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
 				strcmp(te->desc, "SEQUENCE") == 0 ||
 				strcmp(te->desc, "SEQUENCE SET") == 0)
 			{
-				if (!ropt->selTable)
+				if (!ropt->typeopts.selTable)
 					return 0;
 				if (ropt->tableNames.head != NULL &&
 					!simple_string_list_member(&ropt->tableNames, te->tag))
@@ -2949,7 +2949,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
 			}
 			else if (strcmp(te->desc, "INDEX") == 0)
 			{
-				if (!ropt->selIndex)
+				if (!ropt->typeopts.selIndex)
 					return 0;
 				if (ropt->indexNames.head != NULL &&
 					!simple_string_list_member(&ropt->indexNames, te->tag))
@@ -2959,7 +2959,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
 					 strcmp(te->desc, "AGGREGATE") == 0 ||
 					 strcmp(te->desc, "PROCEDURE") == 0)
 			{
-				if (!ropt->selFunction)
+				if (!ropt->typeopts.selFunction)
 					return 0;
 				if (ropt->functionNames.head != NULL &&
 					!simple_string_list_member(&ropt->functionNames, te->tag))
@@ -2967,7 +2967,7 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
 			}
 			else if (strcmp(te->desc, "TRIGGER") == 0)
 			{
-				if (!ropt->selTrigger)
+				if (!ropt->typeopts.selTrigger)
 					return 0;
 				if (ropt->triggerNames.head != NULL &&
 					!simple_string_list_member(&ropt->triggerNames, te->tag))
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 42b8f8d91d5..b4f1ccb2f9b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -114,26 +114,7 @@ static pg_compress_algorithm compression_algorithm = PG_COMPRESSION_NONE;
  * The string lists record the patterns given by command-line switches,
  * which we then convert to lists of OIDs of matching objects.
  */
-static SimpleStringList schema_include_patterns = {NULL, NULL};
-static SimpleOidList schema_include_oids = {NULL, NULL};
-static SimpleStringList schema_exclude_patterns = {NULL, NULL};
-static SimpleOidList schema_exclude_oids = {NULL, NULL};
-
-static SimpleStringList table_include_patterns = {NULL, NULL};
-static SimpleStringList table_include_patterns_and_children = {NULL, NULL};
-static SimpleOidList table_include_oids = {NULL, NULL};
-static SimpleStringList table_exclude_patterns = {NULL, NULL};
-static SimpleStringList table_exclude_patterns_and_children = {NULL, NULL};
-static SimpleOidList table_exclude_oids = {NULL, NULL};
-static SimpleStringList tabledata_exclude_patterns = {NULL, NULL};
-static SimpleStringList tabledata_exclude_patterns_and_children = {NULL, NULL};
-static SimpleOidList tabledata_exclude_oids = {NULL, NULL};
-
-static SimpleStringList foreign_servers_include_patterns = {NULL, NULL};
-static SimpleOidList foreign_servers_include_oids = {NULL, NULL};
-
-static SimpleStringList extension_include_patterns = {NULL, NULL};
-static SimpleOidList extension_include_oids = {NULL, NULL};
+struct FilterOpts filter_opts;
 
 static const CatalogId nilCatalogId = {0, 0};
 
@@ -327,7 +308,6 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
-static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -495,7 +475,7 @@ main(int argc, char **argv)
 				break;
 
 			case 'e':			/* include extension(s) */
-				simple_string_list_append(&extension_include_patterns, optarg);
+				simple_string_list_append(&filter_opts.extension_include_patterns, optarg);
 				dopt.include_everything = false;
 				break;
 
@@ -523,12 +503,12 @@ main(int argc, char **argv)
 				break;
 
 			case 'n':			/* include schema(s) */
-				simple_string_list_append(&schema_include_patterns, optarg);
+				simple_string_list_append(&filter_opts.schema_include_patterns, optarg);
 				dopt.include_everything = false;
 				break;
 
 			case 'N':			/* exclude schema(s) */
-				simple_string_list_append(&schema_exclude_patterns, optarg);
+				simple_string_list_append(&filter_opts.schema_exclude_patterns, optarg);
 				break;
 
 			case 'O':			/* Don't reconnect to match owner */
@@ -552,12 +532,12 @@ main(int argc, char **argv)
 				break;
 
 			case 't':			/* include table(s) */
-				simple_string_list_append(&table_include_patterns, optarg);
+				simple_string_list_append(&filter_opts.table_include_patterns, optarg);
 				dopt.include_everything = false;
 				break;
 
 			case 'T':			/* exclude table(s) */
-				simple_string_list_append(&table_exclude_patterns, optarg);
+				simple_string_list_append(&filter_opts.table_exclude_patterns, optarg);
 				break;
 
 			case 'U':
@@ -600,7 +580,7 @@ main(int argc, char **argv)
 				break;
 
 			case 4:				/* exclude table(s) data */
-				simple_string_list_append(&tabledata_exclude_patterns, optarg);
+				simple_string_list_append(&filter_opts.tabledata_exclude_patterns, optarg);
 				break;
 
 			case 5:				/* section */
@@ -639,28 +619,35 @@ main(int argc, char **argv)
 				break;
 
 			case 11:			/* include foreign data */
-				simple_string_list_append(&foreign_servers_include_patterns,
+				simple_string_list_append(&filter_opts.foreign_servers_include_patterns,
 										  optarg);
 				break;
 
 			case 12:			/* include table(s) and their children */
-				simple_string_list_append(&table_include_patterns_and_children,
+				simple_string_list_append(&filter_opts.table_include_patterns_and_children,
 										  optarg);
 				dopt.include_everything = false;
 				break;
 
 			case 13:			/* exclude table(s) and their children */
-				simple_string_list_append(&table_exclude_patterns_and_children,
+				simple_string_list_append(&filter_opts.table_exclude_patterns_and_children,
 										  optarg);
 				break;
 
 			case 14:			/* exclude data of table(s) and children */
-				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+				simple_string_list_append(&filter_opts.tabledata_exclude_patterns_and_children,
 										  optarg);
 				break;
 
 			case 15:			/* object filters from file */
-				read_dump_filters(optarg, &dopt);
+				{
+					struct type_opts typeopts;
+					// XXX: do something with it....
+					read_filters(progname, optarg, &filter_opts, &typeopts, &dopt.include_everything,
+							FILTER_OBJECT_TYPE_SCHEMA | FILTER_OBJECT_TYPE_TABLE | FILTER_OBJECT_TYPE_FOREIGN_DATA, /* inclusions */
+							FILTER_OBJECT_TYPE_SCHEMA | FILTER_OBJECT_TYPE_TABLE | FILTER_OBJECT_TYPE_TABLE_DATA | /*XXX not really:*/FILTER_OBJECT_TYPE_FOREIGN_DATA /* exclusions */
+						    );
+				}
 				break;
 
 			default:
@@ -701,10 +688,10 @@ main(int argc, char **argv)
 	if (dopt.dataOnly && dopt.schemaOnly)
 		pg_fatal("options -s/--schema-only and -a/--data-only cannot be used together");
 
-	if (dopt.schemaOnly && foreign_servers_include_patterns.head != NULL)
+	if (dopt.schemaOnly && filter_opts.foreign_servers_include_patterns.head != NULL)
 		pg_fatal("options -s/--schema-only and --include-foreign-data cannot be used together");
 
-	if (numWorkers > 1 && foreign_servers_include_patterns.head != NULL)
+	if (numWorkers > 1 && filter_opts.foreign_servers_include_patterns.head != NULL)
 		pg_fatal("option --include-foreign-data is not supported with parallel backup");
 
 	if (dopt.dataOnly && dopt.outputClean)
@@ -828,57 +815,59 @@ main(int argc, char **argv)
 	pg_log_info("last built-in OID is %u", g_last_builtin_oid);
 
 	/* Expand schema selection patterns into OID lists */
-	if (schema_include_patterns.head != NULL)
+	if (filter_opts.schema_include_patterns.head != NULL)
 	{
-		expand_schema_name_patterns(fout, &schema_include_patterns,
-									&schema_include_oids,
+		expand_schema_name_patterns(fout, &filter_opts.schema_include_patterns,
+									&filter_opts.schema_include_oids,
 									strict_names);
-		if (schema_include_oids.head == NULL)
+		if (filter_opts.schema_include_oids.head == NULL)
 			pg_fatal("no matching schemas were found");
 	}
-	expand_schema_name_patterns(fout, &schema_exclude_patterns,
-								&schema_exclude_oids,
+	expand_schema_name_patterns(fout, &filter_opts.schema_exclude_patterns,
+								&filter_opts.schema_exclude_oids,
 								false);
 	/* non-matching exclusion patterns aren't an error */
 
 	/* Expand table selection patterns into OID lists */
-	expand_table_name_patterns(fout, &table_include_patterns,
-							   &table_include_oids,
+	expand_table_name_patterns(fout, &filter_opts.table_include_patterns,
+							   &filter_opts.table_include_oids,
 							   strict_names, false);
-	expand_table_name_patterns(fout, &table_include_patterns_and_children,
-							   &table_include_oids,
+
+	expand_table_name_patterns(fout, &filter_opts.table_include_patterns_and_children,
+							   &filter_opts.table_include_oids,
 							   strict_names, true);
-	if ((table_include_patterns.head != NULL ||
-		 table_include_patterns_and_children.head != NULL) &&
-		table_include_oids.head == NULL)
+
+	if ((filter_opts.table_include_patterns.head != NULL ||
+		filter_opts.table_include_patterns_and_children.head != NULL) &&
+		filter_opts.table_include_oids.head == NULL)
 		pg_fatal("no matching tables were found");
 
-	expand_table_name_patterns(fout, &table_exclude_patterns,
-							   &table_exclude_oids,
+	expand_table_name_patterns(fout, &filter_opts.table_exclude_patterns,
+							   &filter_opts.table_exclude_oids,
 							   false, false);
-	expand_table_name_patterns(fout, &table_exclude_patterns_and_children,
-							   &table_exclude_oids,
+	expand_table_name_patterns(fout, &filter_opts.table_exclude_patterns_and_children,
+							   &filter_opts.table_exclude_oids,
 							   false, true);
 
-	expand_table_name_patterns(fout, &tabledata_exclude_patterns,
-							   &tabledata_exclude_oids,
+	expand_table_name_patterns(fout, &filter_opts.tabledata_exclude_patterns,
+							   &filter_opts.tabledata_exclude_oids,
 							   false, false);
-	expand_table_name_patterns(fout, &tabledata_exclude_patterns_and_children,
-							   &tabledata_exclude_oids,
+	expand_table_name_patterns(fout, &filter_opts.tabledata_exclude_patterns_and_children,
+							   &filter_opts.tabledata_exclude_oids,
 							   false, true);
 
-	expand_foreign_server_name_patterns(fout, &foreign_servers_include_patterns,
-										&foreign_servers_include_oids);
+	expand_foreign_server_name_patterns(fout, &filter_opts.foreign_servers_include_patterns,
+										&filter_opts.foreign_servers_include_oids);
 
 	/* non-matching exclusion patterns aren't an error */
 
 	/* Expand extension selection patterns into OID lists */
-	if (extension_include_patterns.head != NULL)
+	if (filter_opts.extension_include_patterns.head != NULL)
 	{
-		expand_extension_name_patterns(fout, &extension_include_patterns,
-									   &extension_include_oids,
+		expand_extension_name_patterns(fout, &filter_opts.extension_include_patterns,
+									   &filter_opts.extension_include_oids,
 									   strict_names);
-		if (extension_include_oids.head == NULL)
+		if (filter_opts.extension_include_oids.head == NULL)
 			pg_fatal("no matching extensions were found");
 	}
 
@@ -1729,11 +1718,11 @@ selectDumpableNamespace(NamespaceInfo *nsinfo, Archive *fout)
 	 * namespaces. If specific namespaces are being dumped, dump just those
 	 * namespaces. Otherwise, dump all non-system namespaces.
 	 */
-	if (table_include_oids.head != NULL)
+	if (filter_opts.table_include_oids.head != NULL)
 		nsinfo->dobj.dump_contains = nsinfo->dobj.dump = DUMP_COMPONENT_NONE;
-	else if (schema_include_oids.head != NULL)
+	else if (filter_opts.schema_include_oids.head != NULL)
 		nsinfo->dobj.dump_contains = nsinfo->dobj.dump =
-			simple_oid_list_member(&schema_include_oids,
+			simple_oid_list_member(&filter_opts.schema_include_oids,
 								   nsinfo->dobj.catId.oid) ?
 			DUMP_COMPONENT_ALL : DUMP_COMPONENT_NONE;
 	else if (fout->remoteVersion >= 90600 &&
@@ -1782,7 +1771,7 @@ selectDumpableNamespace(NamespaceInfo *nsinfo, Archive *fout)
 	 * In any case, a namespace can be excluded by an exclusion switch
 	 */
 	if (nsinfo->dobj.dump_contains &&
-		simple_oid_list_member(&schema_exclude_oids,
+		simple_oid_list_member(&filter_opts.schema_exclude_oids,
 							   nsinfo->dobj.catId.oid))
 		nsinfo->dobj.dump_contains = nsinfo->dobj.dump = DUMP_COMPONENT_NONE;
 
@@ -1810,8 +1799,8 @@ selectDumpableTable(TableInfo *tbinfo, Archive *fout)
 	 * If specific tables are being dumped, dump just those tables; else, dump
 	 * according to the parent namespace's dump flag.
 	 */
-	if (table_include_oids.head != NULL)
-		tbinfo->dobj.dump = simple_oid_list_member(&table_include_oids,
+	if (filter_opts.table_include_oids.head != NULL)
+		tbinfo->dobj.dump = simple_oid_list_member(&filter_opts.table_include_oids,
 												   tbinfo->dobj.catId.oid) ?
 			DUMP_COMPONENT_ALL : DUMP_COMPONENT_NONE;
 	else
@@ -1821,7 +1810,7 @@ selectDumpableTable(TableInfo *tbinfo, Archive *fout)
 	 * In any case, a table can be excluded by an exclusion switch
 	 */
 	if (tbinfo->dobj.dump &&
-		simple_oid_list_member(&table_exclude_oids,
+		simple_oid_list_member(&filter_opts.table_exclude_oids,
 							   tbinfo->dobj.catId.oid))
 		tbinfo->dobj.dump = DUMP_COMPONENT_NONE;
 }
@@ -2005,9 +1994,9 @@ selectDumpableExtension(ExtensionInfo *extinfo, DumpOptions *dopt)
 	else
 	{
 		/* check if there is a list of extensions to dump */
-		if (extension_include_oids.head != NULL)
+		if (filter_opts.extension_include_oids.head != NULL)
 			extinfo->dobj.dump = extinfo->dobj.dump_contains =
-				simple_oid_list_member(&extension_include_oids,
+				simple_oid_list_member(&filter_opts.extension_include_oids,
 									   extinfo->dobj.catId.oid) ?
 				DUMP_COMPONENT_ALL : DUMP_COMPONENT_NONE;
 		else
@@ -2719,8 +2708,8 @@ makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo)
 		return;
 	/* Skip FOREIGN TABLEs (no data to dump) unless requested explicitly */
 	if (tbinfo->relkind == RELKIND_FOREIGN_TABLE &&
-		(foreign_servers_include_oids.head == NULL ||
-		 !simple_oid_list_member(&foreign_servers_include_oids,
+		(filter_opts.foreign_servers_include_oids.head == NULL ||
+		 !simple_oid_list_member(&filter_opts.foreign_servers_include_oids,
 								 tbinfo->foreign_server)))
 		return;
 	/* Skip partitioned tables (data in partitions) */
@@ -2733,7 +2722,7 @@ makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo)
 		return;
 
 	/* Check that the data is not explicitly excluded */
-	if (simple_oid_list_member(&tabledata_exclude_oids,
+	if (simple_oid_list_member(&filter_opts.tabledata_exclude_oids,
 							   tbinfo->dobj.catId.oid))
 		return;
 
@@ -17849,8 +17838,8 @@ processExtensionTables(Archive *fout, ExtensionInfo extinfo[],
 		 * Check if this extension is listed as to include in the dump.  If
 		 * not, any table data associated with it is discarded.
 		 */
-		if (extension_include_oids.head != NULL &&
-			!simple_oid_list_member(&extension_include_oids,
+		if (filter_opts.extension_include_oids.head != NULL &&
+			!simple_oid_list_member(&filter_opts.extension_include_oids,
 									curext->dobj.catId.oid))
 			continue;
 
@@ -17883,8 +17872,8 @@ processExtensionTables(Archive *fout, ExtensionInfo extinfo[],
 				if (!(curext->dobj.dump & DUMP_COMPONENT_DEFINITION))
 				{
 					/* check table explicitly requested */
-					if (table_include_oids.head != NULL &&
-						simple_oid_list_member(&table_include_oids,
+					if (filter_opts.table_include_oids.head != NULL &&
+						simple_oid_list_member(&filter_opts.table_include_oids,
 											   configtbloid))
 						dumpobj = true;
 
@@ -17895,13 +17884,13 @@ processExtensionTables(Archive *fout, ExtensionInfo extinfo[],
 				}
 
 				/* check table excluded by an exclusion switch */
-				if (table_exclude_oids.head != NULL &&
-					simple_oid_list_member(&table_exclude_oids,
+				if (filter_opts.table_exclude_oids.head != NULL &&
+					simple_oid_list_member(&filter_opts.table_exclude_oids,
 										   configtbloid))
 					dumpobj = false;
 
 				/* check schema excluded by an exclusion switch */
-				if (simple_oid_list_member(&schema_exclude_oids,
+				if (simple_oid_list_member(&filter_opts.schema_exclude_oids,
 										   configtbl->dobj.namespace->dobj.catId.oid))
 					dumpobj = false;
 
@@ -18484,111 +18473,3 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
-
-/*
- * read_dump_filters - retrieve object identifier patterns from file
- *
- * Parse the specified filter file for include and exclude patterns, and add
- * them to the relevant lists.  If the filename is "-" then filters will be
- * read from STDIN rather than a file.
- */
-static void
-read_dump_filters(const char *filename, DumpOptions *dopt)
-{
-	FilterStateData fstate;
-	bool		is_include;
-	char	   *objname;
-	FilterObjectType objtype;
-
-	if (!filter_init(&fstate, filename))
-		exit_nicely(1);
-
-	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
-	{
-		/* ignore comments and empty lines */
-		if (objtype == FILTER_OBJECT_TYPE_NONE)
-			continue;
-
-		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
-		{
-			if (is_include)
-			{
-				log_invalid_filter_format(&fstate,
-										  "\"include\" table data filter is not allowed");
-				break;
-			}
-			else
-				simple_string_list_append(&tabledata_exclude_patterns,
-										  objname);
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
-		{
-			if (is_include)
-			{
-				log_invalid_filter_format(&fstate,
-										  "\"include\" table data and children filter is not allowed");
-				break;
-			}
-			else
-				simple_string_list_append(&tabledata_exclude_patterns_and_children,
-										  objname);
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
-		{
-			if (is_include)
-				simple_string_list_append(&foreign_servers_include_patterns,
-										  objname);
-			else
-			{
-				log_invalid_filter_format(&fstate,
-										  "\"exclude\" foreign data filter is not allowed");
-				break;
-			}
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
-		{
-			if (is_include)
-			{
-				simple_string_list_append(&schema_include_patterns,
-										  objname);
-				dopt->include_everything = false;
-			}
-			else
-				simple_string_list_append(&schema_exclude_patterns,
-										  objname);
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
-		{
-			if (is_include)
-			{
-				simple_string_list_append(&table_include_patterns, objname);
-				dopt->include_everything = false;
-			}
-			else
-				simple_string_list_append(&table_exclude_patterns, objname);
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
-		{
-			if (is_include)
-			{
-				simple_string_list_append(&table_include_patterns_and_children, objname);
-				dopt->include_everything = false;
-			}
-			else
-				simple_string_list_append(&table_exclude_patterns_and_children, objname);
-		}
-		else
-		{
-			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
-			break;
-		}
-
-		if (objname)
-			free(objname);
-	}
-
-	filter_free(&fstate);
-
-	if (fstate.is_error)
-		exit_nicely(1);
-}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index d58eaaaa063..502ab128337 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -82,7 +82,6 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
-static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -120,8 +119,8 @@ static char role_catalog[10];
 static FILE *OPF;
 static char *filename = NULL;
 
-static SimpleStringList database_exclude_patterns = {NULL, NULL};
 static SimpleStringList database_exclude_names = {NULL, NULL};
+struct FilterOpts filter_opts;
 
 #define exit_nicely(code) exit(code)
 
@@ -355,7 +354,7 @@ main(int argc, char *argv[])
 				break;
 
 			case 6:
-				simple_string_list_append(&database_exclude_patterns, optarg);
+				simple_string_list_append(&filter_opts.database_exclude_patterns, optarg);
 				break;
 
 			case 7:
@@ -364,7 +363,12 @@ main(int argc, char *argv[])
 				break;
 
 			case 8:
-				read_dumpall_filters(optarg, &database_exclude_patterns);
+				{
+					bool foo;
+					struct type_opts typeopts;
+					read_filters(progname, optarg, &filter_opts, &typeopts, &foo,
+							0, FILTER_OBJECT_TYPE_DATABASE);
+				}
 				break;
 
 			default:
@@ -383,7 +387,8 @@ main(int argc, char *argv[])
 		exit_nicely(1);
 	}
 
-	if (database_exclude_patterns.head != NULL &&
+	if (filter_opts.database_exclude_patterns.head != NULL &&
+
 		(globals_only || roles_only || tablespaces_only))
 	{
 		pg_log_error("option --exclude-database cannot be used together with -g/--globals-only, -r/--roles-only, or -t/--tablespaces-only");
@@ -495,7 +500,7 @@ main(int argc, char *argv[])
 	/*
 	 * Get a list of database names that match the exclude patterns
 	 */
-	expand_dbname_patterns(conn, &database_exclude_patterns,
+	expand_dbname_patterns(conn, &filter_opts.database_exclude_patterns,
 						   &database_exclude_names);
 
 	/*
@@ -1939,55 +1944,3 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
-
-/*
- * read_dumpall_filters - retrieve database identifier patterns from file
- *
- * Parse the specified filter file for include and exclude patterns, and add
- * them to the relevant lists.  If the filename is "-" then filters will be
- * read from STDIN rather than a file.
- *
- * At the moment, the only allowed filter is for database exclusion.
- */
-static void
-read_dumpall_filters(const char *filename, SimpleStringList *pattern)
-{
-	FilterStateData fstate;
-	bool		is_include;
-	char	   *objname;
-	FilterObjectType objtype;
-
-	if (!filter_init(&fstate, filename))
-		exit_nicely(1);
-
-	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
-	{
-		if (objtype == FILTER_OBJECT_TYPE_NONE)
-			continue;
-
-		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
-		{
-			if (!is_include)
-				simple_string_list_append(pattern, objname);
-			else
-			{
-				log_invalid_filter_format(&fstate,
-										   "\"include\" database filter is not allowed");
-				break;
-			}
-		}
-		else
-		{
-			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
-			break;
-		}
-
-		if (objname)
-			free(objname);
-	}
-
-	filter_free(&fstate);
-
-	if (fstate.is_error)
-		exit_nicely(1);
-}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 0ea71351737..02390886946 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -53,12 +53,12 @@
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
-static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
 {
 	RestoreOptions *opts;
+	struct FilterOpts filter_opts = {0};
 	int			c;
 	int			exit_code;
 	int			numWorkers = 1;
@@ -202,11 +202,11 @@ main(int argc, char **argv)
 				break;
 
 			case 'n':			/* Dump data for this schema only */
-				simple_string_list_append(&opts->schemaNames, optarg);
+				simple_string_list_append(&filter_opts.schema_include_patterns, optarg);
 				break;
 
 			case 'N':			/* Do not dump data for this schema */
-				simple_string_list_append(&opts->schemaExcludeNames, optarg);
+				simple_string_list_append(&filter_opts.schema_exclude_patterns, optarg);
 				break;
 
 			case 'O':
@@ -221,19 +221,19 @@ main(int argc, char **argv)
 				/* no-op, still accepted for backwards compatibility */
 				break;
 			case 'P':			/* Function */
-				opts->selTypes = 1;
-				opts->selFunction = 1;
-				simple_string_list_append(&opts->functionNames, optarg);
+				opts->typeopts.selTypes = 1;
+				opts->typeopts.selFunction = 1;
+				simple_string_list_append(&filter_opts.function_include_patterns, optarg);
 				break;
 			case 'I':			/* Index */
-				opts->selTypes = 1;
-				opts->selIndex = 1;
-				simple_string_list_append(&opts->indexNames, optarg);
+				opts->typeopts.selTypes = 1;
+				opts->typeopts.selIndex = 1;
+				simple_string_list_append(&filter_opts.index_include_patterns, optarg);
 				break;
 			case 'T':			/* Trigger */
-				opts->selTypes = 1;
-				opts->selTrigger = 1;
-				simple_string_list_append(&opts->triggerNames, optarg);
+				opts->typeopts.selTypes = 1;
+				opts->typeopts.selTrigger = 1;
+				simple_string_list_append(&filter_opts.trigger_include_patterns, optarg);
 				break;
 			case 's':			/* dump schema only */
 				opts->schemaOnly = 1;
@@ -243,9 +243,9 @@ main(int argc, char **argv)
 					opts->superuser = pg_strdup(optarg);
 				break;
 			case 't':			/* Dump specified table(s) only */
-				opts->selTypes = 1;
-				opts->selTable = 1;
-				simple_string_list_append(&opts->tableNames, optarg);
+				opts->typeopts.selTypes = 1;
+				opts->typeopts.selTable = 1;
+				simple_string_list_append(&filter_opts.table_include_patterns, optarg);
 				break;
 
 			case 'U':
@@ -290,7 +290,14 @@ main(int argc, char **argv)
 				break;
 
 			case 4:
-				read_restore_filters(optarg, opts);
+				{
+					bool include_everything = opts->include_everything;
+					read_filters(progname, optarg, &filter_opts, &opts->typeopts, &include_everything,
+						FILTER_OBJECT_TYPE_SCHEMA | FILTER_OBJECT_TYPE_TABLE_DATA | FILTER_OBJECT_TYPE_FOREIGN_DATA | FILTER_OBJECT_TYPE_FUNCTION | FILTER_OBJECT_TYPE_INDEX | FILTER_OBJECT_TYPE_TABLE | FILTER_OBJECT_TYPE_TRIGGER, /* allowed to include */
+						FILTER_OBJECT_TYPE_SCHEMA /* allowed to exclude */
+						);
+					opts->include_everything = include_everything;
+				}
 				break;
 
 			default:
@@ -300,6 +307,28 @@ main(int argc, char **argv)
 		}
 	}
 
+	/*
+	 * Any values read into filter_opts are appended to the corresponding
+	 * values in opts, which are used during restore.
+	 */
+	for (SimpleStringListCell *cell = filter_opts.trigger_include_patterns.head; cell; cell = cell->next)
+		simple_string_list_append(&opts->triggerNames, cell->val);
+
+	for (SimpleStringListCell *cell = filter_opts.schema_include_patterns.head; cell; cell = cell->next)
+		simple_string_list_append(&opts->schemaNames, cell->val);
+
+	for (SimpleStringListCell *cell = filter_opts.schema_exclude_patterns.head; cell; cell = cell->next)
+		simple_string_list_append(&opts->schemaExcludeNames, cell->val);
+
+	for (SimpleStringListCell *cell = filter_opts.function_include_patterns.head; cell; cell = cell->next)
+		simple_string_list_append(&opts->functionNames, cell->val);
+
+	for (SimpleStringListCell *cell = filter_opts.index_include_patterns.head; cell; cell = cell->next)
+		simple_string_list_append(&opts->indexNames, cell->val);
+
+	for (SimpleStringListCell *cell = filter_opts.table_include_patterns.head; cell; cell = cell->next)
+		simple_string_list_append(&opts->tableNames, cell->val);
+
 	/* Get file name from command line */
 	if (optind < argc)
 		inputFileSpec = argv[optind++];
@@ -502,109 +531,3 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
-
-/*
- * read_restore_filters - retrieve object identifier patterns from file
- *
- * Parse the specified filter file for include and exclude patterns, and add
- * them to the relevant lists.  If the filename is "-" then filters will be
- * read from STDIN rather than a file.
- */
-static void
-read_restore_filters(const char *filename, RestoreOptions *opts)
-{
-	FilterStateData fstate;
-	bool		is_include;
-	char	   *objname;
-	FilterObjectType objtype;
-
-	if (!filter_init(&fstate, filename))
-		exit_nicely(1);
-
-	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
-	{
-		/* ignore comments or empty lines */
-		if (objtype == FILTER_OBJECT_TYPE_NONE)
-			continue;
-
-		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
-		{
-			if (is_include)
-			{
-				opts->selTypes = 1;
-				opts->selFunction = 1;
-				simple_string_list_append(&opts->functionNames, objname);
-			}
-			else
-			{
-				log_invalid_filter_format(&fstate,
-										   "\"exclude\" function filter is not allowed");
-				break;
-			}
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
-		{
-			if (is_include)
-			{
-				opts->selTypes = 1;
-				opts->selIndex = 1;
-				simple_string_list_append(&opts->indexNames, objname);
-			}
-			else
-			{
-				log_invalid_filter_format(&fstate,
-										   "\"exclude\" index filter is not allowed");
-				break;
-			}
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
-		{
-			if (is_include)
-				simple_string_list_append(&opts->schemaNames, objname);
-			else
-				simple_string_list_append(&opts->schemaExcludeNames, objname);
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
-		{
-			if (is_include)
-			{
-				opts->selTypes = 1;
-				opts->selTable = 1;
-				simple_string_list_append(&opts->tableNames, objname);
-			}
-			else
-			{
-				log_invalid_filter_format(&fstate,
-										   "\"exclude\" table filter is not allowed");
-				break;
-			}
-		}
-		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
-		{
-			if (is_include)
-			{
-				opts->selTypes = 1;
-				opts->selTrigger = 1;
-				simple_string_list_append(&opts->triggerNames, objname);
-			}
-			else
-			{
-				log_invalid_filter_format(&fstate,
-										   "\"exclude\" trigger filter is not allowed");
-				break;
-			}
-		}
-		else
-		{
-			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
-			break;
-		}
-
-		if (objname)
-			free(objname);
-	}
-
-	filter_free(&fstate);
-	if (fstate.is_error)
-		exit_nicely(1);
-}
-- 
2.34.1

#186Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#185)
Re: proposal: possibility to read dumped table's name from file

ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote:

rebase + enhancing about related option from a563c24

Thanks.

It looks like this doesn't currently handle extensions, which were added
at 6568cef26e.

+           <literal>table_and_children</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>, except that
+           it also includes any partitions or inheritance child
+           tables of the table(s) matching the
+           <replaceable class="parameter">pattern</replaceable>.

Why doesn't this just say "works like --table-and-children" ?

I think as you wrote log_invalid_filter_format(), the messages wouldn't
be translated, because they're printed via %s. One option is to call
_() on the message.

+ok($dump !=~ qr/^CREATE TABLE public\.bootab/m, "exclude dumped

children table");

!=~ is being interpretted as as numeric "!=" and throwing warnings.
It should be a !~ b, right ?
It'd be nice if perl warnings during the tests were less easy to miss.

+ * char is not alpha. The char '_' is allowed too (exclude first

position).

Why is it treated specially? Could it be treated the same as alpha?

+                             log_invalid_filter_format(&fstate,
+

"\"include\" table data filter is not allowed");

+                             log_invalid_filter_format(&fstate,
+

"\"include\" table data and children filter is not allowed");

For these, it might be better to write the literal option:

+

"include filter for \"table_data_and_children\" is not allowed");

Because the option is a literal and shouldn't be translated.
And it's probably better to write that using %s, like:

+

"include filter for \"%s\" is not allowed");

That makes shorter and fewer strings.

Find attached a bunch of other corrections as 0002.txt

Thank you very much - I'll recheck the mentioned points tomorrow.

I also dug up what I'd started in november, trying to reduce the code
duplication betwen pg_restore/dump/all. This isn't done, but I might
never finish it, so I'll at least show what I have in case you think
it's a good idea. This passes tests on CI, except for autoconf, due to
using exit_nicely() differently.

Your implementation reduced 60 lines, but the interface and code is more
complex. I cannot say what is significantly better. Personally, in this
case, I prefer my variant, because I think it is a little bit more
readable, and possible modification can be more simple. But this is just my
opinion, and I have no problem accepting other opinions. I can imagine to
define some configuration array like getopt, but it looks like over
engineering

Regards

Pavel

Show quoted text

--
Justin

#187Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#185)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote:

rebase + enhancing about related option from a563c24

Thanks.

It looks like this doesn't currently handle extensions, which were added
at 6568cef26e.

+           <literal>table_and_children</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>, except that
+           it also includes any partitions or inheritance child
+           tables of the table(s) matching the
+           <replaceable class="parameter">pattern</replaceable>.

Why doesn't this just say "works like --table-and-children" ?

changed

I think as you wrote log_invalid_filter_format(), the messages wouldn't
be translated, because they're printed via %s. One option is to call
_() on the message.

fixed

+ok($dump !=~ qr/^CREATE TABLE public\.bootab/m, "exclude dumped

children table");

!=~ is being interpretted as as numeric "!=" and throwing warnings.
It should be a !~ b, right ?
It'd be nice if perl warnings during the tests were less easy to miss.

should be fixed by you

+ * char is not alpha. The char '_' is allowed too (exclude first

position).

Why is it treated specially? Could it be treated the same as alpha?

It is usual behaviour in Postgres for keywords. Important is the complete
sentence "Returns NULL when the buffer is empty or the first char is not
alpha."

In this case this implementation has no big impact on behaviour - probably
you got a message "unknown keyword" instead of "missing keyword". But I
would
implement behaviour consistent with other places. My opinion in this case
is not extra strong - we can define the form of keywords like we want, just
this is consistent
with other parsers in Postgres.

+                             log_invalid_filter_format(&fstate,
+

"\"include\" table data filter is not allowed");

+                             log_invalid_filter_format(&fstate,
+

"\"include\" table data and children filter is not allowed");

For these, it might be better to write the literal option:

+

"include filter for \"table_data_and_children\" is not allowed");

Because the option is a literal and shouldn't be translated.
And it's probably better to write that using %s, like:

+

"include filter for \"%s\" is not allowed");

done

That makes shorter and fewer strings.

Find attached a bunch of other corrections as 0002.txt

merged

Regards

Pavel

Attachments:

v20230320-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20230320-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From 844b54e0dcdb65dbf7f86a75b977db16dad07254 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 107 +++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 509 ++++++++++++++
 src/bin/pg_dump/filter.h                    |  57 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 114 ++++
 src/bin/pg_dump/pg_dumpall.c                |  60 +-
 src/bin/pg_dump/pg_restore.c                | 110 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 701 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1710 insertions(+), 3 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index d6b1faa804..f3e287b75a 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -829,6 +829,99 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { table | table_and_children | schema | foreign_data | table_data | table_data_and_children } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-childrent</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1159,6 +1252,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1581,6 +1675,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858..547fe3803f 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index eb8f59459a..bff55b6b1a 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -30,6 +30,7 @@ OBJS = \
 	compress_lz4.o \
 	compress_none.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -47,8 +48,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..ef77459038
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,509 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("\"%s\" doesn't support filter for object type \"%s\"."),
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * Emit error message "exclude" or "include" filter for filter type
+ * is not allowed.
+ */
+void
+log_unallowed_filter_type(FilterStateData *fstate,
+						  FilterObjectType fot,
+						  bool is_include)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("%s filter for \"%s\" is not allowed."),
+					  is_include ? "include" : "exclude",
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, _("unexpected end of file"));
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, _("missing object name pattern"));
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude"
+ * object_type can one of: "table", "table_and_children", "schema", "foreign_data",
+ * "table_data", "table_and_children", "database", "function", "trigger" or "index"
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   _("no filter command found (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  _("invalid filter command (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, _("missing filter object type"));
+				return false;
+			}
+
+			if (is_keyword_str("table_data", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+			else if (is_keyword_str("table_data_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+			else if (is_keyword_str("database", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_DATABASE;
+			else if (is_keyword_str("foreign_data",keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+			else if (is_keyword_str("function", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+			else if (is_keyword_str("index", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_INDEX;
+			else if (is_keyword_str("schema", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+			else if (is_keyword_str("table", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE;
+			else if (is_keyword_str("table_and_children", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+			else if (is_keyword_str("trigger", keyword, size))
+				*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+			else
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, _("unsupported filter object type: \"%.*s\""), size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..9da8fd4afa
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,57 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+											   const char *appname, FilterObjectType fot);
+extern void log_unallowed_filter_type(FilterStateData *fstate,
+									  FilterObjectType fot, bool is_include);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index b2fb7ac77f..0a626e6cc6 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -6,6 +6,7 @@ pg_dump_common_sources = files(
   'compress_lz4.c',
   'compress_none.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -97,6 +98,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d62780a088..5aaa5b0b35 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -326,6 +327,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -404,6 +406,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 15},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -656,6 +659,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 15:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1102,6 +1109,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18475,3 +18484,108 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children, objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index cd421c5944..c060d96236 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1908,7 +1916,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1932,3 +1939,54 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..71414b27a1 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,105 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..5cb26919b8
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,701 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 96;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index e3ffc653e5..fca7b7d0de 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.40.0

#188Justin Pryzby
pryzby@telsasoft.com
In reply to: Pavel Stehule (#187)
Re: proposal: possibility to read dumped table's name from file

On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote:

ne 19. 3. 2023 v 15:01 odes�latel Justin Pryzby <pryzby@telsasoft.com> napsal:

On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote:

rebase + enhancing about related option from a563c24

Thanks.

It looks like this doesn't currently handle extensions, which were added
at 6568cef26e.

What about this part ? Should extension filters be supported ?

I think the comment that I'd patched that lists all the filter types
should be minimized, rather than duplicating the list of all the
possible filters that's already in the usrr-facing documentation.

One new typo: childrent

#189Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#188)
Re: proposal: possibility to read dumped table's name from file

út 21. 3. 2023 v 16:32 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote:

ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote:

rebase + enhancing about related option from a563c24

Thanks.

It looks like this doesn't currently handle extensions, which were

added

at 6568cef26e.

What about this part ? Should extension filters be supported ?

I missed this, yes, it should be supported.

Show quoted text

I think the comment that I'd patched that lists all the filter types
should be minimized, rather than duplicating the list of all the
possible filters that's already in the usrr-facing documentation.

One new typo: childrent

#190Pavel Stehule
pavel.stehule@gmail.com
In reply to: Justin Pryzby (#188)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

út 21. 3. 2023 v 16:32 odesílatel Justin Pryzby <pryzby@telsasoft.com>
napsal:

On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote:

ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby <pryzby@telsasoft.com>

napsal:

On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote:

rebase + enhancing about related option from a563c24

Thanks.

It looks like this doesn't currently handle extensions, which were

added

at 6568cef26e.

What about this part ? Should extension filters be supported ?

should be fixed

I think the comment that I'd patched that lists all the filter types
should be minimized, rather than duplicating the list of all the
possible filters that's already in the usrr-facing documentation.

I modified this comment. Please, check

One new typo: childrent

fixed

Regards

Pavel

Attachments:

v20230321-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20230321-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From f91f525f32f51f7c5784dd7af57fe2b692db5e7f Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 530 +++++++++++++++
 src/bin/pg_dump/filter.h                    |  58 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 127 ++++
 src/bin/pg_dump/pg_dumpall.c                |  60 +-
 src/bin/pg_dump/pg_restore.c                | 110 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1768 insertions(+), 3 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index d6b1faa804..17bfc661a9 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -829,6 +829,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1159,6 +1259,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1581,6 +1682,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858..547fe3803f 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index eb8f59459a..bff55b6b1a 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -30,6 +30,7 @@ OBJS = \
 	compress_lz4.o \
 	compress_none.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -47,8 +48,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..be50fbf503
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,530 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data",keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("\"%s\" doesn't support filter for object type \"%s\"."),
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * Emit error message "exclude" or "include" filter for filter type
+ * is not allowed.
+ */
+void
+log_unallowed_filter_type(FilterStateData *fstate,
+						  FilterObjectType fot,
+						  bool is_include)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("%s filter for \"%s\" is not allowed."),
+					  is_include ? "include" : "exclude",
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, _("unexpected end of file"));
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, _("missing object name pattern"));
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   _("no filter command found (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  _("invalid filter command (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, _("missing filter object type"));
+				return false;
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, _("unsupported filter object type: \"%.*s\""), size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..9750e6a0d8
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+											   const char *appname, FilterObjectType fot);
+extern void log_unallowed_filter_type(FilterStateData *fstate,
+									  FilterObjectType fot, bool is_include);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index b2fb7ac77f..0a626e6cc6 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -6,6 +6,7 @@ pg_dump_common_sources = files(
   'compress_lz4.c',
   'compress_none.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -97,6 +98,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d62780a088..2b09418ad5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -59,6 +59,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -326,6 +327,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -404,6 +406,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 15},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -656,6 +659,10 @@ main(int argc, char **argv)
 										  optarg);
 				break;
 
+			case 15:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1102,6 +1109,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18475,3 +18484,121 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_EXTENSION)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&extension_include_patterns, objname);
+				dopt->include_everything = false;
+				break;
+			}
+			else
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children,
+										  objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index cd421c5944..c060d96236 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1908,7 +1916,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1932,3 +1939,54 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..71414b27a1 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,105 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..4ca63ca80b
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index e3ffc653e5..fca7b7d0de 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -451,6 +451,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.40.0

#191Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#190)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

only rebase

Regards

Pavel

Attachments:

v20230911-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20230911-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From b62d99f51da03b1fb8dae577fc49420bf36a3bad Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 530 +++++++++++++++
 src/bin/pg_dump/filter.h                    |  58 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 126 ++++
 src/bin/pg_dump/pg_dumpall.c                |  60 +-
 src/bin/pg_dump/pg_restore.c                | 110 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1767 insertions(+), 3 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c1e2220b3c..ae1cc522a8 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -835,6 +835,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1165,6 +1265,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1608,6 +1709,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858..547fe3803f 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 24de7593a6..14765fc8b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..be50fbf503
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,530 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data",keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("\"%s\" doesn't support filter for object type \"%s\"."),
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * Emit error message "exclude" or "include" filter for filter type
+ * is not allowed.
+ */
+void
+log_unallowed_filter_type(FilterStateData *fstate,
+						  FilterObjectType fot,
+						  bool is_include)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("%s filter for \"%s\" is not allowed."),
+					  is_include ? "include" : "exclude",
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, _("unexpected end of file"));
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, _("missing object name pattern"));
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   _("no filter command found (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  _("invalid filter command (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, _("missing filter object type"));
+				return false;
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, _("unsupported filter object type: \"%.*s\""), size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..9750e6a0d8
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+											   const char *appname, FilterObjectType fot);
+extern void log_unallowed_filter_type(FilterStateData *fstate,
+									  FilterObjectType fot, bool is_include);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f7b6176692..9082357933 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -406,6 +408,7 @@ main(int argc, char **argv)
 		{"enable-row-security", no_argument, &dopt.enable_row_security, 1},
 		{"exclude-table-data", required_argument, NULL, 4},
 		{"extra-float-digits", required_argument, NULL, 8},
+		{"filter", required_argument, NULL, 15},
 		{"if-exists", no_argument, &dopt.if_exists, 1},
 		{"inserts", no_argument, NULL, 9},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -662,6 +665,9 @@ main(int argc, char **argv)
 			case 15:
 				if (!parse_sync_method(optarg, &sync_method))
 					exit_nicely(1);
+
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
 				break;
 
 			default:
@@ -1111,6 +1117,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18752,3 +18760,121 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_EXTENSION)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&extension_include_patterns, objname);
+				dopt->include_everything = false;
+				break;
+			}
+			else
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children,
+										  objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..17cf182455 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,54 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..71414b27a1 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,105 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..4ca63ca80b
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 9e05eb91b1..1d92cc676d 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -453,6 +453,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.41.0

#192Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#191)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

po 11. 9. 2023 v 6:34 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

only rebase

Unfortunately this rebase was not correct. I am sorry.

fixed version

Regards

Pavel

Show quoted text

Regards

Pavel

Attachments:

v20230911-2-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20230911-2-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From 32ea3d180ccd976b266bb48c5445c96a1aaf7a54 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 530 +++++++++++++++
 src/bin/pg_dump/filter.h                    |  58 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 127 +++-
 src/bin/pg_dump/pg_dumpall.c                |  60 +-
 src/bin/pg_dump/pg_restore.c                | 110 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1767 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c1e2220b3c..ae1cc522a8 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -835,6 +835,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1165,6 +1265,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1608,6 +1709,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858..547fe3803f 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 24de7593a6..14765fc8b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..be50fbf503
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,530 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data",keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("\"%s\" doesn't support filter for object type \"%s\"."),
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * Emit error message "exclude" or "include" filter for filter type
+ * is not allowed.
+ */
+void
+log_unallowed_filter_type(FilterStateData *fstate,
+						  FilterObjectType fot,
+						  bool is_include)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("%s filter for \"%s\" is not allowed."),
+					  is_include ? "include" : "exclude",
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, _("unexpected end of file"));
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, _("missing object name pattern"));
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   _("no filter command found (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  _("invalid filter command (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, _("missing filter object type"));
+				return false;
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, _("unsupported filter object type: \"%.*s\""), size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..9750e6a0d8
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+											   const char *appname, FilterObjectType fot);
+extern void log_unallowed_filter_type(FilterStateData *fstate,
+									  FilterObjectType fot, bool is_include);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f7b6176692..9f652e4b8b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,7 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
-
+		{"filter", required_argument, NULL, 16},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -662,6 +664,9 @@ main(int argc, char **argv)
 			case 15:
 				if (!parse_sync_method(optarg, &sync_method))
 					exit_nicely(1);
+
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
 				break;
 
 			default:
@@ -1111,6 +1116,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18752,3 +18759,121 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_EXTENSION)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&extension_include_patterns, objname);
+				dopt->include_everything = false;
+				break;
+			}
+			else
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children,
+										  objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..17cf182455 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,54 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..71414b27a1 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,105 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..4ca63ca80b
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 9e05eb91b1..1d92cc676d 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -453,6 +453,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.41.0

#193Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#192)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

po 11. 9. 2023 v 6:57 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

po 11. 9. 2023 v 6:34 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

only rebase

Unfortunately this rebase was not correct. I am sorry.

fixed version

and fixed forgotten "break" in switch

Regards

Pavel

Show quoted text

Regards

Pavel

Regards

Pavel

Attachments:

v20230911-2-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20230911-2-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From 1f0738992d69d0d748bd4494a9244c353c224ace Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 530 +++++++++++++++
 src/bin/pg_dump/filter.h                    |  58 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 128 +++-
 src/bin/pg_dump/pg_dumpall.c                |  60 +-
 src/bin/pg_dump/pg_restore.c                | 110 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1768 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index c1e2220b3c..ae1cc522a8 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -835,6 +835,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1165,6 +1265,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1608,6 +1709,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index e219a79858..547fe3803f 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -122,6 +122,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 47bd7dbda0..ffeb564c52 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -188,6 +188,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 24de7593a6..14765fc8b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..be50fbf503
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,530 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+bool
+filter_init(FilterStateData *fstate, const char *filename)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			return false;
+		}
+	}
+	else
+		fstate->fp = stdin;
+
+	fstate->is_error = false;
+
+	return true;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+static const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data",keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+/*
+ * Emit error message "invalid format in filter file ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+static void
+log_invalid_filter_format(FilterStateData *fstate, char *message)
+{
+	if (fstate->fp != stdin)
+	{
+		pg_log_error("invalid format in filter file \"%s\" on line %d: %s",
+					 fstate->filename,
+					 fstate->lineno,
+					 message);
+	}
+	else
+		pg_log_error("invalid format in filter on line %d: %s",
+					 fstate->lineno,
+					 message);
+
+	fstate->is_error = true;
+}
+
+/*
+ * Emit error message "The application doesn't support filter for object type ..."
+ *
+ * This is mostly a convenience routine to avoid duplicating file closing code
+ * in multiple callsites.
+ */
+void
+log_unsupported_filter_object_type(FilterStateData *fstate,
+									const char *appname,
+									FilterObjectType fot)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("\"%s\" doesn't support filter for object type \"%s\"."),
+					  appname,
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * Emit error message "exclude" or "include" filter for filter type
+ * is not allowed.
+ */
+void
+log_unallowed_filter_type(FilterStateData *fstate,
+						  FilterObjectType fot,
+						  bool is_include)
+{
+	PQExpBuffer str = createPQExpBuffer();
+
+	printfPQExpBuffer(str,
+					  _("%s filter for \"%s\" is not allowed."),
+					  is_include ? "include" : "exclude",
+					  filter_object_type_name(fot));
+
+	log_invalid_filter_format(fstate, str->data);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi lined string.
+ *
+ * Returns pointer to next char after ending double quotes or NULL on error.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+					const char *str,
+					PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+				{
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+					fstate->is_error = true;
+				}
+				else
+					log_invalid_filter_format(fstate, _("unexpected end of file"));
+
+				return NULL;
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier, or NULL on
+ * error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool	skip_space = true;
+	bool	found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		log_invalid_filter_format(fstate, _("missing object name pattern"));
+		return NULL;
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found
+			 * in original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+			if (!str)
+				return NULL;
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	Assert(!fstate->is_error);
+
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate,
+										   _("no filter command found (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				log_invalid_filter_format(fstate,
+										  _("invalid filter command (expected \"include\" or \"exclude\")"));
+				return false;
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				log_invalid_filter_format(fstate, _("missing filter object type"));
+				return false;
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				PQExpBuffer str = createPQExpBuffer();
+
+				printfPQExpBuffer(str, _("unsupported filter object type: \"%.*s\""), size, keyword);
+				log_invalid_filter_format(fstate, str->data);
+				return false;
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			if (!str)
+				return false;
+
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->is_error = true;
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..9750e6a0d8
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	int			lineno;
+	StringInfoData linebuff;
+	bool		is_error;
+}			FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+}			FilterObjectType;
+
+extern bool filter_init(FilterStateData *fstate, const char *filename);
+extern void filter_free(FilterStateData *fstate);
+extern void log_unsupported_filter_object_type(FilterStateData *fstate,
+											   const char *appname, FilterObjectType fot);
+extern void log_unallowed_filter_type(FilterStateData *fstate,
+									  FilterObjectType fot, bool is_include);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f7b6176692..33d90830b8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,7 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
-
+		{"filter", required_argument, NULL, 16},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -664,6 +666,10 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1111,6 +1117,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18752,3 +18760,121 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments and empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+			else
+				simple_string_list_append(&tabledata_exclude_patterns_and_children,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_EXTENSION)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&extension_include_patterns, objname);
+				dopt->include_everything = false;
+				break;
+			}
+			else
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_FOREIGN_DATA)
+		{
+			if (is_include)
+				simple_string_list_append(&foreign_servers_include_patterns,
+										  objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&schema_include_patterns,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&schema_exclude_patterns,
+										  objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns, objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN)
+		{
+			if (is_include)
+			{
+				simple_string_list_append(&table_include_patterns_and_children,
+										  objname);
+				dopt->include_everything = false;
+			}
+			else
+				simple_string_list_append(&table_exclude_patterns_and_children,
+										  objname);
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dump", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..17cf182455 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,54 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_DATABASE)
+		{
+			if (!is_include)
+				simple_string_list_append(pattern, objname);
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_dumpall", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..71414b27a1 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,105 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	if (!filter_init(&fstate, filename))
+		exit_nicely(1);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		/* ignore comments or empty lines */
+		if (objtype == FILTER_OBJECT_TYPE_NONE)
+			continue;
+
+		if (objtype == FILTER_OBJECT_TYPE_FUNCTION)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selFunction = 1;
+				simple_string_list_append(&opts->functionNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_INDEX)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selIndex = 1;
+				simple_string_list_append(&opts->indexNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_SCHEMA)
+		{
+			if (is_include)
+				simple_string_list_append(&opts->schemaNames, objname);
+			else
+				simple_string_list_append(&opts->schemaExcludeNames, objname);
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TABLE)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTable = 1;
+				simple_string_list_append(&opts->tableNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else if (objtype == FILTER_OBJECT_TYPE_TRIGGER)
+		{
+			if (is_include)
+			{
+				opts->selTypes = 1;
+				opts->selTrigger = 1;
+				simple_string_list_append(&opts->triggerNames, objname);
+			}
+			else
+			{
+				log_unallowed_filter_type(&fstate, objtype, is_include);
+				break;
+			}
+		}
+		else
+		{
+			log_unsupported_filter_object_type(&fstate, "pg_restore", objtype);
+			break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+	if (fstate.is_error)
+		exit_nicely(1);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..4ca63ca80b
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/"pg_dumpall" doesn't support filter for object type "table"/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 9e05eb91b1..1d92cc676d 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -453,6 +453,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.41.0

#194Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#193)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

I went and had another look at this. The patch has been around for 18
commitfests and is widely considered to add a good feature, so it seems about
time to get reach closure.

As I've mentioned in the past I'm not a big fan of the parser, but the thread
has overruled on that. Another thing I think is a bit overcomplicated is the
layered error handling for printing log messages, and bubbling up of errors to
get around not being able to call exit_nicely.

In the attached version I've boiled down the error logging into a single new
function pg_log_filter_error() which takes a variable format string. This
removes a fair bit of the extra calls and makes logging easier. I've also
added a function pointer to the FilterStateData for passing the exit function
via filter_init. This allows the filtering code to exit gracefully regardless
of which application is using it. Finally, I've also reimplemented the logic
for checking the parsed tokens into switch statements without defaults in order
to get the compilerwarning on a missed case. It's easy to miss adding code to
handle a state, especially when adding new ones, and this should help highlight
that.

Overall, this does shave a bit off the patch in size for what IMHO is better
readability and maintainability. (I've also made a pgindent pass over it of
course).

What are your thoughts on this version? It's not in a committable state as it
needs a bit more comments here and there and a triplecheck that nothing was
missed in changing this, but I prefer to get your thoughts before spending the
extra time.

--
Daniel Gustafsson

Attachments:

v20231109-0001-possibility-to-read-options-for-dump-from-.patchapplication/octet-stream; name=v20231109-0001-possibility-to-read-options-for-dump-from-.patch; x-unix-mode=0644Download
From faf49822f377b28f9fd10fa8aa88a4e0749c141a Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <okbob@github.com>
Date: Thu, 16 Mar 2023 08:18:08 +0100
Subject: [PATCH v20231109] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 471 +++++++++++++
 src/bin/pg_dump/filter.h                    |  59 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 114 +++-
 src/bin/pg_dump/pg_dumpall.c                |  68 +-
 src/bin/pg_dump/pg_restore.c                | 103 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1697 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8695571045..e2f100d552 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -836,6 +836,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1168,6 +1268,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1611,6 +1712,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index d31585216c..75ba03f3ad 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -125,6 +125,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 374d8d8715..64f7c5dc4d 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -190,6 +190,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 24de7593a6..14765fc8b1 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..46970a57e3
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,471 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+void
+filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	fstate->exit_nicely = f_exit;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			fstate->exit_nicely(1);
+		}
+	}
+	else
+		fstate->fp = stdin;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	if (!fstate)
+		return;
+
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+void
+pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+{
+	va_list		argp;
+	char		buf[256];
+
+	va_start(argp, fmt);
+	vsnprintf(buf, sizeof(buf), fmt, argp);
+	va_end(argp);
+
+	pg_log_error("invalid format in filter \"%s\" on line %d: %s",
+				 (fstate->fp == stdin ? "stdin" : fstate->filename),
+				 fstate->lineno,
+				 buf);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi line string
+ *
+ * Reads a quoted string which can span over multiple lines and returns a
+ * pointer to next char after ending double quotes; it will exit on errors.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+				   const char *str,
+				   PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+				else
+					pg_log_filter_error(fstate, _("unexpected end of file"));
+
+				fstate->exit_nicely(1);
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier and exits
+ * on error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool		skip_space = true;
+	bool		found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		pg_log_filter_error(fstate, _("missing object name pattern"));
+		fstate->exit_nicely(1);
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found in
+			 * original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate,
+									_("no filter command found (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				pg_log_filter_error(fstate,
+									_("invalid filter command (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate, _("missing filter object type"));
+				fstate->exit_nicely(1);
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				pg_log_filter_error(fstate,
+									_("unsupported filter object type: \"%.*s\""), size, keyword);
+				fstate->exit_nicely(1);
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->exit_nicely(1);
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..8e8fc6faee
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,59 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/* Function signature for exit_nicely functions */
+typedef void (*exit_function) (int status);
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	exit_function exit_nicely;
+	int			lineno;
+	StringInfoData linebuff;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+} FilterObjectType;
+
+extern const char *filter_object_type_name(FilterObjectType fot);
+extern void filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit);
+extern void filter_free(FilterStateData *fstate);
+extern void pg_log_filter_error(FilterStateData *fstate, const char *fmt,...);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e863913849..3a4ba7cfbe 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,7 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
-
+		{"filter", required_argument, NULL, 16},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -664,6 +666,10 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1111,6 +1117,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18769,3 +18777,107 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;		/* unreachable */
+
+				case FILTER_OBJECT_TYPE_EXTENSION:
+					simple_string_list_append(&extension_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					simple_string_list_append(&foreign_servers_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_include_patterns_and_children,
+											  objname);
+					dopt->include_everything = false;
+					break;
+			}
+		}
+		else
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;
+
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+					simple_string_list_append(&tabledata_exclude_patterns,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+					simple_string_list_append(&tabledata_exclude_patterns_and_children,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_exclude_patterns_and_children,
+											  objname);
+					break;
+			}
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..6d21128818 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,62 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+								"include",
+								filter_object_type_name(objtype));
+			exit_nicely(1);
+		}
+
+		switch (objtype)
+		{
+			case FILTER_OBJECT_TYPE_NONE:
+				break;
+			case FILTER_OBJECT_TYPE_FUNCTION:
+			case FILTER_OBJECT_TYPE_INDEX:
+			case FILTER_OBJECT_TYPE_TABLE_DATA:
+			case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			case FILTER_OBJECT_TYPE_TRIGGER:
+			case FILTER_OBJECT_TYPE_EXTENSION:
+			case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			case FILTER_OBJECT_TYPE_SCHEMA:
+			case FILTER_OBJECT_TYPE_TABLE:
+			case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				pg_log_filter_error(&fstate, _("unsupported filter object."));
+				exit_nicely(1);
+				break;
+
+			case FILTER_OBJECT_TYPE_DATABASE:
+				simple_string_list_append(pattern, objname);
+				break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..f647bde28d 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,98 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_FUNCTION:
+					opts->selTypes = 1;
+					opts->selFunction = 1;
+					simple_string_list_append(&opts->functionNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_INDEX:
+					opts->selTypes = 1;
+					opts->selIndex = 1;
+					simple_string_list_append(&opts->indexNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					opts->selTypes = 1;
+					opts->selTable = 1;
+					simple_string_list_append(&opts->tableNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					opts->selTypes = 1;
+					opts->selTrigger = 1;
+					simple_string_list_append(&opts->triggerNames, objname);
+					break;
+			}
+		}
+		else
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaExcludeNames, objname);
+					break;
+			}
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a0aee12543
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/pg_dumpall: error: invalid format in filter/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 84f648c174..bcbcd8116f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -455,6 +455,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.32.1 (Apple Git-133)

#195Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#194)
2 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

What are your thoughts on this version? It's not in a committable state
as it
needs a bit more comments here and there and a triplecheck that nothing was
missed in changing this, but I prefer to get your thoughts before spending
the
extra time.

I think using pointer to exit function is an elegant solution. I checked
the code and I found only one issue. I fixed warning

[13:57:22.578] time make -s -j${BUILD_JOBS} world-bin
[13:58:20.858] filter.c: In function ‘pg_log_filter_error’:
[13:58:20.858] filter.c:161:2: error: function ‘pg_log_filter_error’ might
be a candidate for ‘gnu_printf’ format attribute
[-Werror=suggest-attribute=format]
[13:58:20.858] 161 | vsnprintf(buf, sizeof(buf), fmt, argp);
[13:58:20.858] | ^~~~~~~~~
[13:58:20.858] cc1: all warnings being treated as errors

and probably copy/paste bug

diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f647bde28d..ab2abedf5f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -535,7 +535,7 @@ read_restore_filters(const char *filename,
RestoreOptions *opts)
                case FILTER_OBJECT_TYPE_EXTENSION:
                case FILTER_OBJECT_TYPE_FOREIGN_DATA:
                    pg_log_filter_error(&fstate, _("%s filter for \"%s\" is
not allowed."),
-                                       "exclude",
+                                       "include",
                                        filter_object_type_name(objtype));
                    exit_nicely(1);

Regards

Pavel

Show quoted text

--
Daniel Gustafsson

Attachments:

v20231112-0002-fix-err-message.patchtext/x-patch; charset=US-ASCII; name=v20231112-0002-fix-err-message.patchDownload
From 9be8fb14a3fe75aa4203a059e8372986bf5e6615 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Sun, 12 Nov 2023 13:54:26 +0100
Subject: [PATCH 2/2] fix err message

---
 src/bin/pg_dump/pg_restore.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f647bde28d..ab2abedf5f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -535,7 +535,7 @@ read_restore_filters(const char *filename, RestoreOptions *opts)
 				case FILTER_OBJECT_TYPE_EXTENSION:
 				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
 					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
-										"exclude",
+										"include",
 										filter_object_type_name(objtype));
 					exit_nicely(1);
 
-- 
2.41.0

v20231112-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20231112-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From cbaae854eca0cc88bb0886abfd45416ad13cffc7 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Sat, 11 Nov 2023 20:34:34 +0100
Subject: [PATCH 1/2] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 471 +++++++++++++
 src/bin/pg_dump/filter.h                    |  60 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 114 +++-
 src/bin/pg_dump/pg_dumpall.c                |  68 +-
 src/bin/pg_dump/pg_restore.c                | 103 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1698 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8695571045..e2f100d552 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -836,6 +836,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1168,6 +1268,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1611,6 +1712,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index d31585216c..75ba03f3ad 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -125,6 +125,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 374d8d8715..64f7c5dc4d 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -190,6 +190,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 604cddb997..2bcf2a7002 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..46970a57e3
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,471 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+void
+filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	fstate->exit_nicely = f_exit;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			fstate->exit_nicely(1);
+		}
+	}
+	else
+		fstate->fp = stdin;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	if (!fstate)
+		return;
+
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+void
+pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+{
+	va_list		argp;
+	char		buf[256];
+
+	va_start(argp, fmt);
+	vsnprintf(buf, sizeof(buf), fmt, argp);
+	va_end(argp);
+
+	pg_log_error("invalid format in filter \"%s\" on line %d: %s",
+				 (fstate->fp == stdin ? "stdin" : fstate->filename),
+				 fstate->lineno,
+				 buf);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi line string
+ *
+ * Reads a quoted string which can span over multiple lines and returns a
+ * pointer to next char after ending double quotes; it will exit on errors.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+				   const char *str,
+				   PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+				else
+					pg_log_filter_error(fstate, _("unexpected end of file"));
+
+				fstate->exit_nicely(1);
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier and exits
+ * on error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool		skip_space = true;
+	bool		found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		pg_log_filter_error(fstate, _("missing object name pattern"));
+		fstate->exit_nicely(1);
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found in
+			 * original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate,
+									_("no filter command found (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				pg_log_filter_error(fstate,
+									_("invalid filter command (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate, _("missing filter object type"));
+				fstate->exit_nicely(1);
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				pg_log_filter_error(fstate,
+									_("unsupported filter object type: \"%.*s\""), size, keyword);
+				fstate->exit_nicely(1);
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->exit_nicely(1);
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..cd5c2cf5f7
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/* Function signature for exit_nicely functions */
+typedef void (*exit_function) (int status);
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	exit_function exit_nicely;
+	int			lineno;
+	StringInfoData linebuff;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+} FilterObjectType;
+
+extern const char *filter_object_type_name(FilterObjectType fot);
+extern void filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit);
+extern void filter_free(FilterStateData *fstate);
+extern void pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+  pg_attribute_printf(2, 3);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e863913849..3a4ba7cfbe 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,7 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
-
+		{"filter", required_argument, NULL, 16},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -664,6 +666,10 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1111,6 +1117,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18769,3 +18777,107 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;		/* unreachable */
+
+				case FILTER_OBJECT_TYPE_EXTENSION:
+					simple_string_list_append(&extension_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					simple_string_list_append(&foreign_servers_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_include_patterns_and_children,
+											  objname);
+					dopt->include_everything = false;
+					break;
+			}
+		}
+		else
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;
+
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+					simple_string_list_append(&tabledata_exclude_patterns,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+					simple_string_list_append(&tabledata_exclude_patterns_and_children,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_exclude_patterns_and_children,
+											  objname);
+					break;
+			}
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..6d21128818 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,62 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+								"include",
+								filter_object_type_name(objtype));
+			exit_nicely(1);
+		}
+
+		switch (objtype)
+		{
+			case FILTER_OBJECT_TYPE_NONE:
+				break;
+			case FILTER_OBJECT_TYPE_FUNCTION:
+			case FILTER_OBJECT_TYPE_INDEX:
+			case FILTER_OBJECT_TYPE_TABLE_DATA:
+			case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			case FILTER_OBJECT_TYPE_TRIGGER:
+			case FILTER_OBJECT_TYPE_EXTENSION:
+			case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			case FILTER_OBJECT_TYPE_SCHEMA:
+			case FILTER_OBJECT_TYPE_TABLE:
+			case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				pg_log_filter_error(&fstate, _("unsupported filter object."));
+				exit_nicely(1);
+				break;
+
+			case FILTER_OBJECT_TYPE_DATABASE:
+				simple_string_list_append(pattern, objname);
+				break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..f647bde28d 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,98 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_FUNCTION:
+					opts->selTypes = 1;
+					opts->selFunction = 1;
+					simple_string_list_append(&opts->functionNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_INDEX:
+					opts->selTypes = 1;
+					opts->selIndex = 1;
+					simple_string_list_append(&opts->indexNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					opts->selTypes = 1;
+					opts->selTable = 1;
+					simple_string_list_append(&opts->tableNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					opts->selTypes = 1;
+					opts->selTrigger = 1;
+					simple_string_list_append(&opts->triggerNames, objname);
+					break;
+			}
+		}
+		else
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaExcludeNames, objname);
+					break;
+			}
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a0aee12543
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/pg_dumpall: error: invalid format in filter/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 84f648c174..bcbcd8116f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -455,6 +455,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.41.0

#196Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#195)
4 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

ne 12. 11. 2023 v 14:17 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

Hi

What are your thoughts on this version? It's not in a committable state
as it
needs a bit more comments here and there and a triplecheck that nothing
was
missed in changing this, but I prefer to get your thoughts before
spending the
extra time.

I think using pointer to exit function is an elegant solution. I checked
the code and I found only one issue. I fixed warning

[13:57:22.578] time make -s -j${BUILD_JOBS} world-bin
[13:58:20.858] filter.c: In function ‘pg_log_filter_error’:
[13:58:20.858] filter.c:161:2: error: function ‘pg_log_filter_error’ might
be a candidate for ‘gnu_printf’ format attribute
[-Werror=suggest-attribute=format]
[13:58:20.858] 161 | vsnprintf(buf, sizeof(buf), fmt, argp);
[13:58:20.858] | ^~~~~~~~~
[13:58:20.858] cc1: all warnings being treated as errors

and probably copy/paste bug

diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f647bde28d..ab2abedf5f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -535,7 +535,7 @@ read_restore_filters(const char *filename,
RestoreOptions *opts)
case FILTER_OBJECT_TYPE_EXTENSION:
case FILTER_OBJECT_TYPE_FOREIGN_DATA:
pg_log_filter_error(&fstate, _("%s filter for \"%s\"
is not allowed."),
-                                       "exclude",
+                                       "include",
filter_object_type_name(objtype));
exit_nicely(1);

Regards

Pavel

next update - fix used, but uninitialized "is_include" variable, when
filter is of FILTER_OBJECT_TYPE_NONE

fix crash

# Running: pg_ctl -w -D
/tmp/cirrus-ci-build/build-32/testrun/pg_dump/005_pg_dump_filterfile/data/t_005_pg_dump_filterfile_main_data/pgdata
-l /tmp/cirrus-ci-build/build-32/testrun/pg_dump/005_pg_dump_filterfile/log/005_pg_dump_filterfile_main.log
-o --cluster-name=main start
waiting for server to start.... done
server started
# Postmaster PID for node "main" is 71352
# Running: pg_dump -p 65454 -f
/tmp/cirrus-ci-build/build-32/testrun/pg_dump/005_pg_dump_filterfile/data/t_005_pg_dump_filterfile_main_data/backup/plain.sql
--filter=/tmp/cirrus-ci-build/build-32/testrun/pg_dump/005_pg_dump_filterfile/data/tmp_test_0mO3/inputfile.txt
postgres
../src/bin/pg_dump/pg_dump.c:18800:7: runtime error: load of value 86,
which is not a valid value for type '_Bool'
==71579==Using libbacktrace symbolizer.
#0 0x566302cd in read_dump_filters ../src/bin/pg_dump/pg_dump.c:18800
#1 0x56663429 in main ../src/bin/pg_dump/pg_dump.c:670
#2 0xf7694e45 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x1ae45)
#3 0x56624d50 in _start
(/tmp/cirrus-ci-build/build-32/tmp_install/usr/local/pgsql/bin/pg_dump+0x1ad50)

Regards

Pavel

Show quoted text

--
Daniel Gustafsson

Attachments:

v20231113-0004-fix-uninitialize-is_include-variable-runtime-error-l.patchtext/x-patch; charset=US-ASCII; name=v20231113-0004-fix-uninitialize-is_include-variable-runtime-error-l.patchDownload
From 235bcd7944e99468c3fe6a65ca5490883b983a70 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Mon, 13 Nov 2023 14:10:49 +0100
Subject: [PATCH 4/4] fix uninitialize is_include variable, runtime error: load
 of value 86, which is not a valid value for type '_Bool'

---
 src/bin/pg_dump/filter.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 46970a57e3..2d724b1b35 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -455,6 +455,7 @@ filter_read_item(FilterStateData *fstate,
 		else
 		{
 			*objname = NULL;
+			*is_include = false;
 			*objtype = FILTER_OBJECT_TYPE_NONE;
 		}
 
-- 
2.41.0

v20231113-0003-add-more-tests-related-to-unsupported-options-of-pg_.patchtext/x-patch; charset=US-ASCII; name=v20231113-0003-add-more-tests-related-to-unsupported-options-of-pg_.patchDownload
From 13984bc6ca65b20c4a23a65602304ab0bf11c955 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Mon, 13 Nov 2023 08:26:22 +0100
Subject: [PATCH 3/4] add more tests related to unsupported options of
 pg_restore

---
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 58 ++++++++++++++++++++-
 1 file changed, 57 insertions(+), 1 deletion(-)

diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
index a0aee12543..09d3262b8b 100644
--- a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -6,7 +6,7 @@ use warnings;
 
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 98;
+use Test::More tests => 106;
 
 my $tempdir = PostgreSQL::Test::Utils::tempdir;;
 my $inputfile;
@@ -535,6 +535,62 @@ $dump = slurp_file($plainfile);
 ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
 ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
 
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table_data xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/include filter for "table data" is not allowed/,
+	"invalid syntax: inclusion of non allowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/include filter for "extension" is not allowed/,
+	"invalid syntax: inclusion of non allowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude extension xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/exclude filter for "extension" is not allowed/,
+	"invalid syntax: exclusion of non allowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table_data xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/xclude filter for "table data" is not allowed/,
+	"invalid syntax: exclusion of non allowed object"
+);
+
 #########################################
 # test restore of other objects
 
-- 
2.41.0

v20231113-0002-fix-err-message.patchtext/x-patch; charset=US-ASCII; name=v20231113-0002-fix-err-message.patchDownload
From 0f9ca3ebcbfd27f78c30e873002081b1354d2d1f Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Sun, 12 Nov 2023 13:54:26 +0100
Subject: [PATCH 2/4] fix err message

---
 src/bin/pg_dump/pg_restore.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f647bde28d..ab2abedf5f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -535,7 +535,7 @@ read_restore_filters(const char *filename, RestoreOptions *opts)
 				case FILTER_OBJECT_TYPE_EXTENSION:
 				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
 					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
-										"exclude",
+										"include",
 										filter_object_type_name(objtype));
 					exit_nicely(1);
 
-- 
2.41.0

v20231113-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20231113-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From a7a598b40948a5788f509c630f664fa16861e215 Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Sat, 11 Nov 2023 20:34:34 +0100
Subject: [PATCH 1/4] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 ++++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 471 +++++++++++++
 src/bin/pg_dump/filter.h                    |  60 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 114 +++-
 src/bin/pg_dump/pg_dumpall.c                |  68 +-
 src/bin/pg_dump/pg_restore.c                | 103 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 717 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1698 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8695571045..e2f100d552 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -836,6 +836,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1168,6 +1268,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1611,6 +1712,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index d31585216c..75ba03f3ad 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -125,6 +125,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 374d8d8715..64f7c5dc4d 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -190,6 +190,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 604cddb997..2bcf2a7002 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..46970a57e3
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,471 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+void
+filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	fstate->exit_nicely = f_exit;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			fstate->exit_nicely(1);
+		}
+	}
+	else
+		fstate->fp = stdin;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	if (!fstate)
+		return;
+
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+void
+pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+{
+	va_list		argp;
+	char		buf[256];
+
+	va_start(argp, fmt);
+	vsnprintf(buf, sizeof(buf), fmt, argp);
+	va_end(argp);
+
+	pg_log_error("invalid format in filter \"%s\" on line %d: %s",
+				 (fstate->fp == stdin ? "stdin" : fstate->filename),
+				 fstate->lineno,
+				 buf);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi line string
+ *
+ * Reads a quoted string which can span over multiple lines and returns a
+ * pointer to next char after ending double quotes; it will exit on errors.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+				   const char *str,
+				   PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+				else
+					pg_log_filter_error(fstate, _("unexpected end of file"));
+
+				fstate->exit_nicely(1);
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier and exits
+ * on error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool		skip_space = true;
+	bool		found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		pg_log_filter_error(fstate, _("missing object name pattern"));
+		fstate->exit_nicely(1);
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found in
+			 * original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 bool *is_include,
+				 char **objname,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate,
+									_("no filter command found (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*is_include = true;
+			else if (is_keyword_str("exclude", keyword, size))
+				*is_include = false;
+			else
+			{
+				pg_log_filter_error(fstate,
+									_("invalid filter command (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate, _("missing filter object type"));
+				fstate->exit_nicely(1);
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				pg_log_filter_error(fstate,
+									_("unsupported filter object type: \"%.*s\""), size, keyword);
+				fstate->exit_nicely(1);
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->exit_nicely(1);
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..cd5c2cf5f7
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/* Function signature for exit_nicely functions */
+typedef void (*exit_function) (int status);
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	exit_function exit_nicely;
+	int			lineno;
+	StringInfoData linebuff;
+} FilterStateData;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER
+} FilterObjectType;
+
+extern const char *filter_object_type_name(FilterObjectType fot);
+extern void filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit);
+extern void filter_free(FilterStateData *fstate);
+extern void pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+  pg_attribute_printf(2, 3);
+extern bool filter_read_item(FilterStateData *fstate, bool *is_include,
+							 char **objname, FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e863913849..3a4ba7cfbe 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,7 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
-
+		{"filter", required_argument, NULL, 16},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -664,6 +666,10 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1111,6 +1117,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18769,3 +18777,107 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;		/* unreachable */
+
+				case FILTER_OBJECT_TYPE_EXTENSION:
+					simple_string_list_append(&extension_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					simple_string_list_append(&foreign_servers_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_include_patterns_and_children,
+											  objname);
+					dopt->include_everything = false;
+					break;
+			}
+		}
+		else
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;
+
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+					simple_string_list_append(&tabledata_exclude_patterns,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+					simple_string_list_append(&tabledata_exclude_patterns_and_children,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_exclude_patterns_and_children,
+											  objname);
+					break;
+			}
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..6d21128818 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,62 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+								"include",
+								filter_object_type_name(objtype));
+			exit_nicely(1);
+		}
+
+		switch (objtype)
+		{
+			case FILTER_OBJECT_TYPE_NONE:
+				break;
+			case FILTER_OBJECT_TYPE_FUNCTION:
+			case FILTER_OBJECT_TYPE_INDEX:
+			case FILTER_OBJECT_TYPE_TABLE_DATA:
+			case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			case FILTER_OBJECT_TYPE_TRIGGER:
+			case FILTER_OBJECT_TYPE_EXTENSION:
+			case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			case FILTER_OBJECT_TYPE_SCHEMA:
+			case FILTER_OBJECT_TYPE_TABLE:
+			case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				pg_log_filter_error(&fstate, _("unsupported filter object."));
+				exit_nicely(1);
+				break;
+
+			case FILTER_OBJECT_TYPE_DATABASE:
+				simple_string_list_append(pattern, objname);
+				break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..f647bde28d 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,98 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	bool		is_include;
+	char	   *objname;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &is_include, &objname, &objtype))
+	{
+		if (is_include)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_FUNCTION:
+					opts->selTypes = 1;
+					opts->selFunction = 1;
+					simple_string_list_append(&opts->functionNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_INDEX:
+					opts->selTypes = 1;
+					opts->selIndex = 1;
+					simple_string_list_append(&opts->indexNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					opts->selTypes = 1;
+					opts->selTable = 1;
+					simple_string_list_append(&opts->tableNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					opts->selTypes = 1;
+					opts->selTrigger = 1;
+					simple_string_list_append(&opts->triggerNames, objname);
+					break;
+			}
+		}
+		else
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaExcludeNames, objname);
+					break;
+			}
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..a0aee12543
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,717 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 98;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/pg_dumpall: error: invalid format in filter/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 84f648c174..bcbcd8116f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -455,6 +455,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.41.0

#197Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#196)
Re: proposal: possibility to read dumped table's name from file

On 13 Nov 2023, at 14:15, Pavel Stehule <pavel.stehule@gmail.com> wrote:

Hi

ne 12. 11. 2023 v 14:17 odesílatel Pavel Stehule <pavel.stehule@gmail.com <mailto:pavel.stehule@gmail.com>> napsal:
Hi

What are your thoughts on this version? It's not in a committable state as it
needs a bit more comments here and there and a triplecheck that nothing was
missed in changing this, but I prefer to get your thoughts before spending the
extra time.

I think using pointer to exit function is an elegant solution. I checked the code and I found only one issue. I fixed warning

[13:57:22.578] time make -s -j${BUILD_JOBS} world-bin
[13:58:20.858] filter.c: In function ‘pg_log_filter_error’:
[13:58:20.858] filter.c:161:2: error: function ‘pg_log_filter_error’ might be a candidate for ‘gnu_printf’ format attribute [-Werror=suggest-attribute=format]
[13:58:20.858] 161 | vsnprintf(buf, sizeof(buf), fmt, argp);
[13:58:20.858] | ^~~~~~~~~
[13:58:20.858] cc1: all warnings being treated as errors

and probably copy/paste bug

diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f647bde28d..ab2abedf5f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -535,7 +535,7 @@ read_restore_filters(const char *filename, RestoreOptions *opts)
case FILTER_OBJECT_TYPE_EXTENSION:
case FILTER_OBJECT_TYPE_FOREIGN_DATA:
pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
-                                       "exclude",
+                                       "include",
filter_object_type_name(objtype));
exit_nicely(1);

Regards

Pavel

next update - fix used, but uninitialized "is_include" variable, when filter is of FILTER_OBJECT_TYPE_NONE

Thanks, the posted patchset was indeed a bit of a sketch, thanks for fixing up
these. I'll go over it again too to clean it up and try to make into something
committable.

I was pondering replacing the is_include handling with returning an enum for
the operation, to keep things more future proof in case we add more operations
(and also a bit less magic IMHO).

--
Daniel Gustafsson

#198Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#197)
Re: proposal: possibility to read dumped table's name from file

po 13. 11. 2023 v 14:39 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 13 Nov 2023, at 14:15, Pavel Stehule <pavel.stehule@gmail.com> wrote:

Hi

ne 12. 11. 2023 v 14:17 odesílatel Pavel Stehule <

pavel.stehule@gmail.com <mailto:pavel.stehule@gmail.com>> napsal:

Hi

What are your thoughts on this version? It's not in a committable state

as it

needs a bit more comments here and there and a triplecheck that nothing

was

missed in changing this, but I prefer to get your thoughts before

spending the

extra time.

I think using pointer to exit function is an elegant solution. I checked

the code and I found only one issue. I fixed warning

[13:57:22.578] time make -s -j${BUILD_JOBS} world-bin
[13:58:20.858] filter.c: In function ‘pg_log_filter_error’:
[13:58:20.858] filter.c:161:2: error: function ‘pg_log_filter_error’

might be a candidate for ‘gnu_printf’ format attribute
[-Werror=suggest-attribute=format]

[13:58:20.858] 161 | vsnprintf(buf, sizeof(buf), fmt, argp);
[13:58:20.858] | ^~~~~~~~~
[13:58:20.858] cc1: all warnings being treated as errors

and probably copy/paste bug

diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f647bde28d..ab2abedf5f 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -535,7 +535,7 @@ read_restore_filters(const char *filename,

RestoreOptions *opts)

case FILTER_OBJECT_TYPE_EXTENSION:
case FILTER_OBJECT_TYPE_FOREIGN_DATA:
pg_log_filter_error(&fstate, _("%s filter for \"%s\"

is not allowed."),

-                                       "exclude",
+                                       "include",

filter_object_type_name(objtype));

exit_nicely(1);

Regards

Pavel

next update - fix used, but uninitialized "is_include" variable, when

filter is of FILTER_OBJECT_TYPE_NONE

Thanks, the posted patchset was indeed a bit of a sketch, thanks for
fixing up
these. I'll go over it again too to clean it up and try to make into
something
committable.

I was pondering replacing the is_include handling with returning an enum
for
the operation, to keep things more future proof in case we add more
operations
(and also a bit less magic IMHO).

+1

Pavel

Show quoted text

--
Daniel Gustafsson

#199Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#198)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

I was pondering replacing the is_include handling with returning an enum for

the operation, to keep things more future proof in case we add more
operations
(and also a bit less magic IMHO).

+1

I did it.

Regards

Pavel

Show quoted text

Pavel

--
Daniel Gustafsson

Attachments:

v20231120-0001-possibility-to-read-options-for-dump-from-file.patchtext/x-patch; charset=US-ASCII; name=v20231120-0001-possibility-to-read-options-for-dump-from-file.patchDownload
From 24631830e190cce814ee60a8050126ebc29b7ffa Mon Sep 17 00:00:00 2001
From: "okbob@github.com" <pavel.stehule@gmail.com>
Date: Sat, 11 Nov 2023 20:34:34 +0100
Subject: [PATCH] possibility to read options for dump from file

---
 doc/src/sgml/ref/pg_dump.sgml               | 114 +++
 doc/src/sgml/ref/pg_dumpall.sgml            |  22 +
 doc/src/sgml/ref/pg_restore.sgml            |  25 +
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 472 ++++++++++++
 src/bin/pg_dump/filter.h                    |  70 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 119 ++-
 src/bin/pg_dump/pg_dumpall.c                |  68 +-
 src/bin/pg_dump/pg_restore.c                | 108 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 773 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1775 insertions(+), 4 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8695571045..e2f100d552 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -836,6 +836,106 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           <option>--include-foreign-data</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like
+           <option>-t</option>/<option>--table</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables, works like
+           <option>--table-and-children</option>
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data, works like
+           <option>--exclude-table-data</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any
+           partitions or inheritance child, works like
+           <option>--exclude-table-data-and-children</option>. This keyword can only be
+           used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like
+           <option>-n</option>/<option>--schema</option>
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after filter as well. Blank lines
+        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
+        perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1168,6 +1268,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1611,6 +1712,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables with names starting with mytable, except for table
+   <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index d31585216c..75ba03f3ad 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -125,6 +125,28 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpretted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for excluding databases,
+        and can also be specified more than once for multiple filter files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database  <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 374d8d8715..64f7c5dc4d 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -190,6 +190,31 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpretted according to the
+        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
+        <option>--function</option>, <option>--index</option>, <option>--table</option>
+        or <option>--trigger</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 604cddb997..2bcf2a7002 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..9b5e45c722
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,472 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
+ * different from the one in pg_dumpall, so instead of calling exit_nicely we
+ * have to return some error flag (in this case NULL), and exit_nicely will be
+ * executed from caller's routine.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ * Returns true on success.
+ */
+void
+filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	fstate->exit_nicely = f_exit;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			fstate->exit_nicely(1);
+		}
+	}
+	else
+		fstate->fp = stdin;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	if (!fstate)
+		return;
+
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. It is designed for formatting
+ * of error message in log_unsupported_filter_object_type routine.
+ */
+const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+void
+pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+{
+	va_list		argp;
+	char		buf[256];
+
+	va_start(argp, fmt);
+	vsnprintf(buf, sizeof(buf), fmt, argp);
+	va_end(argp);
+
+	pg_log_error("invalid format in filter \"%s\" on line %d: %s",
+				 (fstate->fp == stdin ? "stdin" : fstate->filename),
+				 fstate->lineno,
+				 buf);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returnlength preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi line string
+ *
+ * Reads a quoted string which can span over multiple lines and returns a
+ * pointer to next char after ending double quotes; it will exit on errors.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+				   const char *str,
+				   PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+				else
+					pg_log_filter_error(fstate, _("unexpected end of file"));
+
+				fstate->exit_nicely(1);
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier and exits
+ * on error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool		skip_space = true;
+	bool		found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		pg_log_filter_error(fstate, _("missing object name pattern"));
+		fstate->exit_nicely(1);
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found in
+			 * original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message before returning
+ * false.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 char **objname,
+				 FilterCommandType *comtype,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate,
+									_("no filter command found (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*comtype = FILTER_COMMAND_TYPE_INCLUDE;
+			else if (is_keyword_str("exclude", keyword, size))
+				*comtype = FILTER_COMMAND_TYPE_EXCLUDE;
+			else
+			{
+				pg_log_filter_error(fstate,
+									_("invalid filter command (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate, _("missing filter object type"));
+				fstate->exit_nicely(1);
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				pg_log_filter_error(fstate,
+									_("unsupported filter object type: \"%.*s\""), size, keyword);
+				fstate->exit_nicely(1);
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*comtype = FILTER_COMMAND_TYPE_NONE;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->exit_nicely(1);
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..4c12f8a572
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,70 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/* Function signature for exit_nicely functions */
+typedef void (*exit_function) (int status);
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	exit_function exit_nicely;
+	int			lineno;
+	StringInfoData linebuff;
+} FilterStateData;
+
+/*
+ * List of command types that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_COMMAND_TYPE_NONE,
+	FILTER_COMMAND_TYPE_INCLUDE,
+	FILTER_COMMAND_TYPE_EXCLUDE,
+} FilterCommandType;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER,
+} FilterObjectType;
+
+extern const char *filter_object_type_name(FilterObjectType fot);
+extern void filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit);
+extern void filter_free(FilterStateData *fstate);
+extern void pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+  pg_attribute_printf(2, 3);
+extern bool filter_read_item(FilterStateData *fstate, char **objname,
+							 FilterCommandType *comtype,FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 34fd0a86e9..1e7756beba 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,7 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
-
+		{"filter", required_argument, NULL, 16},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -664,6 +666,10 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
+			case 16:			/* object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1111,6 +1117,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
+			 "                               in specified file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18771,3 +18779,112 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	char	   *objname;
+	FilterCommandType comtype;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &objname, &comtype, &objtype))
+	{
+		if (comtype == FILTER_COMMAND_TYPE_INCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;		/* unreachable */
+
+				case FILTER_OBJECT_TYPE_EXTENSION:
+					simple_string_list_append(&extension_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					simple_string_list_append(&foreign_servers_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_include_patterns_and_children,
+											  objname);
+					dopt->include_everything = false;
+					break;
+			}
+		}
+		else if (comtype == FILTER_COMMAND_TYPE_EXCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;
+
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+					simple_string_list_append(&tabledata_exclude_patterns,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+					simple_string_list_append(&tabledata_exclude_patterns_and_children,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_exclude_patterns_and_children,
+											  objname);
+					break;
+			}
+		}
+		else
+		{
+			Assert(comtype == FILTER_COMMAND_TYPE_NONE);
+			Assert(objtype == FILTER_OBJECT_TYPE_NONE);
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..fe2a541c56 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -158,6 +160,7 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
+		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1913,7 +1921,6 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
-
 /*
  * dumpTimestamp
  */
@@ -1937,3 +1944,62 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	char	   *objname;
+	FilterCommandType comtype;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit);
+
+	while (filter_read_item(&fstate, &objname, &comtype, &objtype))
+	{
+		if (comtype == FILTER_COMMAND_TYPE_INCLUDE)
+		{
+			pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+								"include",
+								filter_object_type_name(objtype));
+			exit_nicely(1);
+		}
+
+		switch (objtype)
+		{
+			case FILTER_OBJECT_TYPE_NONE:
+				break;
+			case FILTER_OBJECT_TYPE_FUNCTION:
+			case FILTER_OBJECT_TYPE_INDEX:
+			case FILTER_OBJECT_TYPE_TABLE_DATA:
+			case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			case FILTER_OBJECT_TYPE_TRIGGER:
+			case FILTER_OBJECT_TYPE_EXTENSION:
+			case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			case FILTER_OBJECT_TYPE_SCHEMA:
+			case FILTER_OBJECT_TYPE_TABLE:
+			case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				pg_log_filter_error(&fstate, _("unsupported filter object."));
+				exit_nicely(1);
+				break;
+
+			case FILTER_OBJECT_TYPE_DATABASE:
+				simple_string_list_append(pattern, objname);
+				break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..0cca9ee612 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,7 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +502,103 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	char	   *objname;
+	FilterCommandType comtype;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &objname, &comtype, &objtype))
+	{
+		if (comtype == FILTER_COMMAND_TYPE_INCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_FUNCTION:
+					opts->selTypes = 1;
+					opts->selFunction = 1;
+					simple_string_list_append(&opts->functionNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_INDEX:
+					opts->selTypes = 1;
+					opts->selIndex = 1;
+					simple_string_list_append(&opts->indexNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					opts->selTypes = 1;
+					opts->selTable = 1;
+					simple_string_list_append(&opts->tableNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					opts->selTypes = 1;
+					opts->selTrigger = 1;
+					simple_string_list_append(&opts->triggerNames, objname);
+					break;
+			}
+		}
+		else if (comtype == FILTER_COMMAND_TYPE_EXCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaExcludeNames, objname);
+					break;
+			}
+		}
+		else
+		{
+			Assert(comtype == FILTER_COMMAND_TYPE_NONE);
+			Assert(objtype == FILTER_OBJECT_TYPE_NONE);
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..09d3262b8b
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,773 @@
+
+# Copyright (c) 2021, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 106;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching mattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/pg_dumpall: error: invalid format in filter/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table_data xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/include filter for "table data" is not allowed/,
+	"invalid syntax: inclusion of non allowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/include filter for "extension" is not allowed/,
+	"invalid syntax: inclusion of non allowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude extension xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/exclude filter for "extension" is not allowed/,
+	"invalid syntax: exclusion of non allowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table_data xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/xclude filter for "table data" is not allowed/,
+	"invalid syntax: exclusion of non allowed object"
+);
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 84f648c174..bcbcd8116f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -455,6 +455,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.42.0

#200Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#199)
2 attachment(s)
Re: proposal: possibility to read dumped table's name from file

On 20 Nov 2023, at 06:20, Pavel Stehule <pavel.stehule@gmail.com> wrote:

I was pondering replacing the is_include handling with returning an enum for
the operation, to keep things more future proof in case we add more operations
(and also a bit less magic IMHO).

+1

I did it.

Nice, I think it's an improvement.

+           <literal>extension</literal>: data on foreign servers, works like
+           <option>--extension</option>. This keyword can only be
+           used with the <literal>include</literal> keyword.
This seems like a copy-pasteo, fixed in the attached.

I've spent some time polishing this version of the patch, among other things
trying to make the docs and --help screen consistent across the tools. I've
added the diff as a txt file to this email (to keep the CFbot from applying
it), it's mainly reformatting a few comments and making things consistent.

The attached is pretty close to a committable patch IMO, review is welcome on
both the patch and commit message. I tried to identify all reviewers over the
past 3+ years but I might have missed someone.

--
Daniel Gustafsson

Attachments:

v20231121_fixups.txttext/plain; name=v20231121_fixups.txt; x-unix-mode=0644Download
commit 4a3c0bdaf3fd21b75e17244691fbeb9340e960e1
Author: Daniel Gustafsson <dgustafsson@postgresql.org>
Date:   Tue Nov 21 15:08:27 2023 +0100

    Fixups and tweaks

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index e2f100d552..0e5ba4f712 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -873,49 +873,52 @@ PostgreSQL documentation
         <itemizedlist>
          <listitem>
           <para>
-           <literal>extension</literal>: data on foreign servers, works like
-           <option>--extension</option>. This keyword can only be
+           <literal>extension</literal>: extensions, works like the
+           <option>--extension</option> option. This keyword can only be
            used with the <literal>include</literal> keyword.
           </para>
          </listitem>
          <listitem>
           <para>
            <literal>foreign_data</literal>: data on foreign servers, works like
-           <option>--include-foreign-data</option>. This keyword can only be
-           used with the <literal>include</literal> keyword.
+           the <option>--include-foreign-data</option> option. This keyword can
+           only be used with the <literal>include</literal> keyword.
           </para>
          </listitem>
          <listitem>
           <para>
-           <literal>table</literal>: tables, works like
-           <option>-t</option>/<option>--table</option>
+           <literal>table</literal>: tables, works like the
+           <option>-t</option>/<option>--table</option> option.
           </para>
          </listitem>
          <listitem>
           <para>
-           <literal>table_and_children</literal>: tables, works like
-           <option>--table-and-children</option>
+           <literal>table_and_children</literal>: tables including any partitions
+           or inheritance child tables, works like the
+           <option>--table-and-children</option> option.
           </para>
          </listitem>
          <listitem>
           <para>
-           <literal>table_data</literal>: table data, works like
-           <option>--exclude-table-data</option>. This keyword can only be
-           used with the <literal>exclude</literal> keyword.
+           <literal>table_data</literal>: table data of any tables matching
+           <replaceable>pattern</replaceable>, works like the
+           <option>--exclude-table-data</option> option. This keyword can only
+           be used with the <literal>exclude</literal> keyword.
           </para>
          </listitem>
          <listitem>
           <para>
-           <literal>table_data_and_children</literal>: table data of any
-           partitions or inheritance child, works like
-           <option>--exclude-table-data-and-children</option>. This keyword can only be
-           used with the <literal>exclude</literal> keyword.
+           <literal>table_data_and_children</literal>: table data of any tables
+           matching <replaceable>pattern</replaceable> as well as any partitions
+           or inheritance children of the table(s), works like the
+           <option>--exclude-table-data-and-children</option> option. This
+           keyword can only be used with the <literal>exclude</literal> keyword.
           </para>
          </listitem>
          <listitem>
           <para>
-           <literal>schema</literal>: schemas, works like
-           <option>-n</option>/<option>--schema</option>
+           <literal>schema</literal>: schemas, works like the
+           <option>-n</option>/<option>--schema</option> option.
           </para>
          </listitem>
         </itemizedlist>
@@ -923,9 +926,9 @@ PostgreSQL documentation
 
        <para>
         Lines starting with <literal>#</literal> are considered comments and
-        ignored. Comments can be placed after filter as well. Blank lines
-        are also ignored. See <xref linkend="app-psql-patterns"/> for how to
-        perform quoting in patterns.
+        ignored. Comments can be placed after an object pattern row as well.
+        Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+        for how to perform quoting in patterns.
        </para>
 
        <para>
@@ -1715,8 +1718,8 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 </screen></para>
 
   <para>
-   To dump all tables with names starting with mytable, except for table
-   <literal>mytable2</literal>, specify a filter file
+   To dump all tables whose names start with <literal>mytable</literal>, except
+   for table <literal>mytable2</literal>, specify a filter file
    <filename>filter.txt</filename> like:
 <programlisting>
 include table mytable*
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 75ba03f3ad..4d7c046468 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -130,20 +130,29 @@ PostgreSQL documentation
       <listitem>
        <para>
         Specify a filename from which to read patterns for databases excluded
-        from the dump. The patterns are interpretted according to the same rules
+        from the dump. The patterns are interpreted according to the same rules
         as <option>--exclude-database</option>.
         To read from <literal>STDIN</literal>, use <filename>-</filename> as the
         filename.  The <option>--filter</option> option can be specified in
-        conjunction with the above listed options for excluding databases,
-        and can also be specified more than once for multiple filter files.
+        conjunction with <option>--exclude-database</option> for excluding
+        databases, and can also be specified more than once for multiple filter
+        files.
        </para>
 
        <para>
         The file lists one database pattern per row, with the following format:
 <synopsis>
-exclude database  <replaceable class="parameter">PATTERN</replaceable>
+exclude database <replaceable class="parameter">PATTERN</replaceable>
 </synopsis>
        </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after an object pattern row as well.
+        Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+        for how to perform quoting in patterns.
+       </para>
+
       </listitem>
      </varlistentry>
 
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 64f7c5dc4d..1a23874da6 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -195,10 +195,14 @@ PostgreSQL documentation
       <listitem>
        <para>
         Specify a filename from which to read patterns for objects excluded
-        or included from restore. The patterns are interpretted according to the
-        same rules as <option>--schema</option>, <option>--exclude-schema</option>,
-        <option>--function</option>, <option>--index</option>, <option>--table</option>
-        or <option>--trigger</option>.
+        or included from restore. The patterns are interpreted according to the
+        same rules as
+        <option>-n</option>/<option>--schema</option> for including objects in schemas,
+        <option>-N</option>/<option>--exclude-schema</option>for excluding objects in schemas,
+        <option>-P</option>/<option>--function</option> for restoring named functions,
+        <option>-I</option>/<option>--index</option> for restoring named indexes,
+        <option>-t</option>/<option>--table</option> for restoring named tables
+        or <option>-T</option>/<option>--trigger</option> for restoring triggers.
         To read from <literal>STDIN</literal>, use <filename>-</filename> as the
         filename.  The <option>--filter</option> option can be specified in
         conjunction with the above listed options for including or excluding
@@ -212,6 +216,57 @@ PostgreSQL documentation
 { include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
 </synopsis>
        </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>function</literal>: functions, works like the
+           <option>-P</option>/<option>--function</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>index</literal>: indexes, works like the
+           <option>-I</option>/<option>--indexes</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like the
+           <option>-n</option>/<option>--schema</option> and
+           <option>-N</option>/<option>--exclude-schema</option> options.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like the
+           <option>-t</option>/<option>--table</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>trigger</literal>: triggers, works like the
+           <option>-T</option>/<option>--trigger</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after an object pattern row as well.
+        Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+        for how to perform quoting in patterns.
+       </para>
+
       </listitem>
      </varlistentry>
 
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 9b5e45c722..b64822b6b7 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -23,15 +23,13 @@
 
 /*
  * Following routines are called from pg_dump, pg_dumpall and pg_restore.
- * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore is
- * different from the one in pg_dumpall, so instead of calling exit_nicely we
- * have to return some error flag (in this case NULL), and exit_nicely will be
- * executed from caller's routine.
+ * Since the implementation of exit_nicely is application specific, each
+ * application need to pass a function pointer to the exit_nicely function to
+ * use for exiting on errors.
  */
 
 /*
  * Opens filter's file and initialize fstate structure.
- * Returns true on success.
  */
 void
 filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit)
@@ -76,8 +74,8 @@ filter_free(FilterStateData *fstate)
 }
 
 /*
- * Translate FilterObjectType enum to string. It is designed for formatting
- * of error message in log_unsupported_filter_object_type routine.
+ * Translate FilterObjectType enum to string. The main purpose is for error
+ * message formatting.
  */
 const char *
 filter_object_type_name(FilterObjectType fot)
@@ -181,7 +179,7 @@ filter_get_keyword(const char **line, int *size)
 	const char *ptr = *line;
 	const char *result = NULL;
 
-	/* Set returnlength preemptively in case no keyword is found */
+	/* Set returned length preemptively in case no keyword is found */
 	*size = 0;
 
 	/* Skip initial whitespace */
@@ -382,8 +380,7 @@ read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
  * Returns true when one filter item was successfully read and parsed.  When
  * object name contains \n chars, then more than one line from input file can
  * be processed.  Returns false when the filter file reaches EOF. In case of
- * error, the function will emit an appropriate error message before returning
- * false.
+ * error, the function will emit an appropriate error message and exit.
  */
 bool
 filter_read_item(FilterStateData *fstate,
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1e7756beba..64e2d754d1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -436,6 +436,7 @@ main(int argc, char **argv)
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
 		{"filter", required_argument, NULL, 16},
+
 		{NULL, 0, NULL, 0}
 	};
 
@@ -666,7 +667,7 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
-			case 16:			/* object filters from file */
+			case 16:			/* read object filters from file */
 				read_dump_filters(optarg, &dopt);
 				break;
 
@@ -1117,8 +1118,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
-	printf(_("  --filter=FILENAME            dump objects and data based on the filter expressions\n"
-			 "                               in specified file\n"));
+	printf(_("  --filter=FILENAME            include or exclude objects and data from dump\n"
+			 "                               based expressions in FILENAME\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index fe2a541c56..1b974cf7e8 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -160,7 +160,6 @@ main(int argc, char *argv[])
 		{"disable-triggers", no_argument, &disable_triggers, 1},
 		{"exclude-database", required_argument, NULL, 6},
 		{"extra-float-digits", required_argument, NULL, 5},
-		{"filter", required_argument, NULL, 8},
 		{"if-exists", no_argument, &if_exists, 1},
 		{"inserts", no_argument, &inserts, 1},
 		{"lock-wait-timeout", required_argument, NULL, 2},
@@ -180,6 +179,7 @@ main(int argc, char *argv[])
 		{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
 		{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
 		{"rows-per-insert", required_argument, NULL, 7},
+		{"filter", required_argument, NULL, 8},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -660,7 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
-	printf(_("  --filter=FILENAME            exclude databases specified in filter file\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in FILENAME\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1921,6 +1921,7 @@ executeCommand(PGconn *conn, const char *query)
 	PQclear(res);
 }
 
+
 /*
  * dumpTimestamp
  */
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 0cca9ee612..1459e02263 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -470,7 +470,8 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
-	printf(_("  --filter=FILE                restore objects based on filter expressions\n"));
+	printf(_("  --filter=FILENAME            restore or skip objects based on expressions\n"
+			 "                               in FILENAME\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
index 09d3262b8b..d4c2c2340d 100644
--- a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -1,12 +1,12 @@
 
-# Copyright (c) 2021, PostgreSQL Global Development Group
+# Copyright (c) 2023, PostgreSQL Global Development Group
 
 use strict;
 use warnings;
 
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 106;
+use Test::More;
 
 my $tempdir = PostgreSQL::Test::Utils::tempdir;;
 my $inputfile;
@@ -421,7 +421,7 @@ command_ok(
 		"--filter=$tempdir/inputfile.txt",
 		'--strict-names', 'postgres'
 	],
-	"strict names with matching mattern");
+	"strict names with matching pattern");
 
 $dump = slurp_file($plainfile);
 
@@ -546,7 +546,7 @@ command_fails_like(
 		"--filter=$tempdir/inputfile.txt"
 	],
 	qr/include filter for "table data" is not allowed/,
-	"invalid syntax: inclusion of non allowed object"
+	"invalid syntax: inclusion of unallowed object"
 );
 
 open $inputfile, '>', "$tempdir/inputfile.txt"
@@ -560,7 +560,7 @@ command_fails_like(
 		"--filter=$tempdir/inputfile.txt"
 	],
 	qr/include filter for "extension" is not allowed/,
-	"invalid syntax: inclusion of non allowed object"
+	"invalid syntax: inclusion of unallowed object"
 );
 
 open $inputfile, '>', "$tempdir/inputfile.txt"
@@ -574,7 +574,7 @@ command_fails_like(
 		"--filter=$tempdir/inputfile.txt"
 	],
 	qr/exclude filter for "extension" is not allowed/,
-	"invalid syntax: exclusion of non allowed object"
+	"invalid syntax: exclusion of unallowed object"
 );
 
 open $inputfile, '>', "$tempdir/inputfile.txt"
@@ -587,8 +587,8 @@ command_fails_like(
 		'pg_restore', '-p', $port, '-f', $plainfile,
 		"--filter=$tempdir/inputfile.txt"
 	],
-	qr/xclude filter for "table data" is not allowed/,
-	"invalid syntax: exclusion of non allowed object"
+	qr/exclude filter for "table data" is not allowed/,
+	"invalid syntax: exclusion of unallowed object"
 );
 
 #########################################
@@ -771,3 +771,6 @@ command_fails_like(
 	],
 	qr/pg_dump: error: no matching extensions were found/,
 	"dump nonexisting extension");
+
+
+done_testing();
v20231121-0001-Read-include-exclude-commands-for-dump-res.patchapplication/octet-stream; name=v20231121-0001-Read-include-exclude-commands-for-dump-res.patch; x-unix-mode=0644Download
From 25677519edadbda3d07407f69063e4c61a32bad5 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Tue, 21 Nov 2023 15:14:31 +0100
Subject: [PATCH v20231121] Read include/exclude commands for dump/restore from
 file

When there is a need to filter multiple tables with include and/or exclude
options it's not impossible to run into the limitations of the commandline.
This adds a --filter=FILENAME feature to pg_dump, pg_dumpall and pg_restore
which is used to supply a file containing object exclude/include commands
which work just like their commandline counterparts. The format of the file
is one objectpattern per row like:

    <command> <object> <objectpattern>

<command> can be "include" or "exclude" and <object> can be table_data,
table_data_and_children, database, extension, foreign_data, function,
index, schema, table, table_and_children or trigger.

This patch has been through a lot of revisions and design changes over a
long period of time, so the list of reviewers reflect reviewers of some
version, not necessarily the final version.

Written by Pavel Stehule with some additional hackery by me.

Author: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAFj8pRB10wvW0CC9Xq=1XDs=zCQxer3cbLcNZa+qiX4cUH-G_A@mail.gmail.com
---
 doc/src/sgml/ref/pg_dump.sgml               | 117 +++
 doc/src/sgml/ref/pg_dumpall.sgml            |  31 +
 doc/src/sgml/ref/pg_restore.sgml            |  80 ++
 src/bin/pg_dump/Makefile                    |   5 +-
 src/bin/pg_dump/filter.c                    | 469 ++++++++++++
 src/bin/pg_dump/filter.h                    |  70 ++
 src/bin/pg_dump/meson.build                 |   2 +
 src/bin/pg_dump/pg_dump.c                   | 118 +++
 src/bin/pg_dump/pg_dumpall.c                |  67 ++
 src/bin/pg_dump/pg_restore.c                | 109 +++
 src/bin/pg_dump/t/005_pg_dump_filterfile.pl | 776 ++++++++++++++++++++
 src/tools/msvc/Mkvcbuild.pm                 |   1 +
 12 files changed, 1843 insertions(+), 2 deletions(-)
 create mode 100644 src/bin/pg_dump/filter.c
 create mode 100644 src/bin/pg_dump/filter.h
 create mode 100644 src/bin/pg_dump/t/005_pg_dump_filterfile.pl

diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 8695571045..0e5ba4f712 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -836,6 +836,109 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects to include
+        or exclude from the dump. The patterns are interpreted according to the
+        same rules as the corresponding options:
+        <option>-t</option>/<option>--table</option>,
+        <option>--table-and-children</option>,
+        <option>--exclude-table-and-children</option> or
+        <option>-T</option> for tables,
+        <option>-n</option>/<option>--schema</option> for schemas,
+        <option>--include-foreign-data</option> for data on foreign servers and
+        <option>--exclude-table-data</option>,
+        <option>--exclude-table-data-and-children</option> for table data,
+        <option>-e</option>/<option>--extension</option> for extensions.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one object pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>extension</literal>: extensions, works like the
+           <option>--extension</option> option. This keyword can only be
+           used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>foreign_data</literal>: data on foreign servers, works like
+           the <option>--include-foreign-data</option> option. This keyword can
+           only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like the
+           <option>-t</option>/<option>--table</option> option.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_and_children</literal>: tables including any partitions
+           or inheritance child tables, works like the
+           <option>--table-and-children</option> option.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data</literal>: table data of any tables matching
+           <replaceable>pattern</replaceable>, works like the
+           <option>--exclude-table-data</option> option. This keyword can only
+           be used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table_data_and_children</literal>: table data of any tables
+           matching <replaceable>pattern</replaceable> as well as any partitions
+           or inheritance children of the table(s), works like the
+           <option>--exclude-table-data-and-children</option> option. This
+           keyword can only be used with the <literal>exclude</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like the
+           <option>-n</option>/<option>--schema</option> option.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after an object pattern row as well.
+        Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+        for how to perform quoting in patterns.
+       </para>
+
+       <para>
+        Example files are listed below in the <xref linkend="pg-dump-examples"/>
+        section.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>--if-exists</option></term>
       <listitem>
@@ -1168,6 +1271,7 @@ PostgreSQL documentation
         schema (<option>-n</option>/<option>--schema</option>) and
         table (<option>-t</option>/<option>--table</option>) pattern
         match at least one extension/schema/table in the database to be dumped.
+        This also applies to filters used with <option>--filter</option>.
         Note that if none of the extension/schema/table patterns find
         matches, <application>pg_dump</application> will generate an error
         even without <option>--strict-names</option>.
@@ -1611,6 +1715,19 @@ CREATE DATABASE foo WITH TEMPLATE template0;
 
 <screen>
 <prompt>$</prompt> <userinput>pg_dump -t "\"MixedCaseName\"" mydb &gt; mytab.sql</userinput>
+</screen></para>
+
+  <para>
+   To dump all tables whose names start with <literal>mytable</literal>, except
+   for table <literal>mytable2</literal>, specify a filter file
+   <filename>filter.txt</filename> like:
+<programlisting>
+include table mytable*
+exclude table mytable2
+</programlisting>
+
+<screen>
+<prompt>$</prompt> <userinput>pg_dump --filter=filter.txt mydb &gt; db.sql</userinput>
 </screen></para>
 
  </refsect1>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index d31585216c..4d7c046468 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -125,6 +125,37 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for databases excluded
+        from the dump. The patterns are interpreted according to the same rules
+        as <option>--exclude-database</option>.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with <option>--exclude-database</option> for excluding
+        databases, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+exclude database <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after an object pattern row as well.
+        Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+        for how to perform quoting in patterns.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-g</option></term>
       <term><option>--globals-only</option></term>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 374d8d8715..1a23874da6 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -190,6 +190,86 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry>
+      <term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
+      <listitem>
+       <para>
+        Specify a filename from which to read patterns for objects excluded
+        or included from restore. The patterns are interpreted according to the
+        same rules as
+        <option>-n</option>/<option>--schema</option> for including objects in schemas,
+        <option>-N</option>/<option>--exclude-schema</option>for excluding objects in schemas,
+        <option>-P</option>/<option>--function</option> for restoring named functions,
+        <option>-I</option>/<option>--index</option> for restoring named indexes,
+        <option>-t</option>/<option>--table</option> for restoring named tables
+        or <option>-T</option>/<option>--trigger</option> for restoring triggers.
+        To read from <literal>STDIN</literal>, use <filename>-</filename> as the
+        filename.  The <option>--filter</option> option can be specified in
+        conjunction with the above listed options for including or excluding
+        objects, and can also be specified more than once for multiple filter
+        files.
+       </para>
+
+       <para>
+        The file lists one database pattern per row, with the following format:
+<synopsis>
+{ include | exclude } { function | index | schema | table | trigger } <replaceable class="parameter">PATTERN</replaceable>
+</synopsis>
+       </para>
+
+       <para>
+        The first keyword specifies whether the objects matched by the pattern
+        are to be included or excluded. The second keyword specifies the type
+        of object to be filtered using the pattern:
+        <itemizedlist>
+         <listitem>
+          <para>
+           <literal>function</literal>: functions, works like the
+           <option>-P</option>/<option>--function</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>index</literal>: indexes, works like the
+           <option>-I</option>/<option>--indexes</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>schema</literal>: schemas, works like the
+           <option>-n</option>/<option>--schema</option> and
+           <option>-N</option>/<option>--exclude-schema</option> options.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>table</literal>: tables, works like the
+           <option>-t</option>/<option>--table</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <literal>trigger</literal>: triggers, works like the
+           <option>-T</option>/<option>--trigger</option> option. This keyword
+           can only be used with the <literal>include</literal> keyword.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+
+       <para>
+        Lines starting with <literal>#</literal> are considered comments and
+        ignored. Comments can be placed after an object pattern row as well.
+        Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+        for how to perform quoting in patterns.
+       </para>
+
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-F <replaceable class="parameter">format</replaceable></option></term>
       <term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 604cddb997..2bcf2a7002 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -32,6 +32,7 @@ OBJS = \
 	compress_none.o \
 	compress_zstd.o \
 	dumputils.o \
+	filter.o \
 	parallel.o \
 	pg_backup_archiver.o \
 	pg_backup_custom.o \
@@ -49,8 +50,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
 pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
 	$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
-pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o filter.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+	$(CC) $(CFLAGS) pg_dumpall.o dumputils.o filter.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
 
 install: all installdirs
 	$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
new file mode 100644
index 0000000000..b64822b6b7
--- /dev/null
+++ b/src/bin/pg_dump/filter.c
@@ -0,0 +1,469 @@
+/*-------------------------------------------------------------------------
+ *
+ * Implementation of simple filter file parser
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "common/string.h"
+#include "filter.h"
+#include "lib/stringinfo.h"
+#include "pqexpbuffer.h"
+
+#define		is_keyword_str(cstr, str, bytes) \
+	((strlen(cstr) == (bytes)) && (pg_strncasecmp((cstr), (str), (bytes)) == 0))
+
+/*
+ * Following routines are called from pg_dump, pg_dumpall and pg_restore.
+ * Since the implementation of exit_nicely is application specific, each
+ * application need to pass a function pointer to the exit_nicely function to
+ * use for exiting on errors.
+ */
+
+/*
+ * Opens filter's file and initialize fstate structure.
+ */
+void
+filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit)
+{
+	fstate->filename = filename;
+	fstate->lineno = 0;
+	fstate->exit_nicely = f_exit;
+	initStringInfo(&fstate->linebuff);
+
+	if (strcmp(filename, "-") != 0)
+	{
+		fstate->fp = fopen(filename, "r");
+		if (!fstate->fp)
+		{
+			pg_log_error("could not open filter file \"%s\": %m", filename);
+			fstate->exit_nicely(1);
+		}
+	}
+	else
+		fstate->fp = stdin;
+}
+
+/*
+ * Release allocated resources for the given filter.
+ */
+void
+filter_free(FilterStateData *fstate)
+{
+	if (!fstate)
+		return;
+
+	free(fstate->linebuff.data);
+	fstate->linebuff.data = NULL;
+
+	if (fstate->fp && fstate->fp != stdin)
+	{
+		if (fclose(fstate->fp) != 0)
+			pg_log_error("could not close filter file \"%s\": %m", fstate->filename);
+
+		fstate->fp = NULL;
+	}
+}
+
+/*
+ * Translate FilterObjectType enum to string. The main purpose is for error
+ * message formatting.
+ */
+const char *
+filter_object_type_name(FilterObjectType fot)
+{
+	switch (fot)
+	{
+		case FILTER_OBJECT_TYPE_NONE:
+			return "comment or empty line";
+		case FILTER_OBJECT_TYPE_TABLE_DATA:
+			return "table data";
+		case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			return "table data and children";
+		case FILTER_OBJECT_TYPE_DATABASE:
+			return "database";
+		case FILTER_OBJECT_TYPE_EXTENSION:
+			return "extension";
+		case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			return "foreign data";
+		case FILTER_OBJECT_TYPE_FUNCTION:
+			return "function";
+		case FILTER_OBJECT_TYPE_INDEX:
+			return "index";
+		case FILTER_OBJECT_TYPE_SCHEMA:
+			return "schema";
+		case FILTER_OBJECT_TYPE_TABLE:
+			return "table";
+		case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+			return "table and children";
+		case FILTER_OBJECT_TYPE_TRIGGER:
+			return "trigger";
+	}
+
+	/* should never get here */
+	pg_unreachable();
+}
+
+/*
+ * Returns true when keyword is one of supported object types, and
+ * set related objtype. Returns false, when keyword is not assigned
+ * with known object type.
+ */
+static bool
+get_object_type(const char *keyword, int size, FilterObjectType *objtype)
+{
+	if (is_keyword_str("table_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA;
+	else if (is_keyword_str("table_data_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN;
+	else if (is_keyword_str("database", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_DATABASE;
+	else if (is_keyword_str("extension", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_EXTENSION;
+	else if (is_keyword_str("foreign_data", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FOREIGN_DATA;
+	else if (is_keyword_str("function", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_FUNCTION;
+	else if (is_keyword_str("index", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_INDEX;
+	else if (is_keyword_str("schema", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_SCHEMA;
+	else if (is_keyword_str("table", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE;
+	else if (is_keyword_str("table_and_children", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN;
+	else if (is_keyword_str("trigger", keyword, size))
+		*objtype = FILTER_OBJECT_TYPE_TRIGGER;
+	else
+		return false;
+
+	return true;
+}
+
+
+void
+pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+{
+	va_list		argp;
+	char		buf[256];
+
+	va_start(argp, fmt);
+	vsnprintf(buf, sizeof(buf), fmt, argp);
+	va_end(argp);
+
+	pg_log_error("invalid format in filter \"%s\" on line %d: %s",
+				 (fstate->fp == stdin ? "stdin" : fstate->filename),
+				 fstate->lineno,
+				 buf);
+}
+
+/*
+ * filter_get_keyword - read the next filter keyword from buffer
+ *
+ * Search for keywords (limited to ascii alphabetic characters) in
+ * the passed in line buffer. Returns NULL when the buffer is empty or the first
+ * char is not alpha. The char '_' is allowed, except as the first character.
+ * The length of the found keyword is returned in the size parameter.
+ */
+static const char *
+filter_get_keyword(const char **line, int *size)
+{
+	const char *ptr = *line;
+	const char *result = NULL;
+
+	/* Set returned length preemptively in case no keyword is found */
+	*size = 0;
+
+	/* Skip initial whitespace */
+	while (isspace(*ptr))
+		ptr++;
+
+	if (isalpha(*ptr))
+	{
+		result = ptr++;
+
+		while (isalpha(*ptr) || *ptr == '_')
+			ptr++;
+
+		*size = ptr - result;
+	}
+
+	*line = ptr;
+
+	return result;
+}
+
+/*
+ * read_quoted_pattern - read quoted possibly multi line string
+ *
+ * Reads a quoted string which can span over multiple lines and returns a
+ * pointer to next char after ending double quotes; it will exit on errors.
+ */
+static const char *
+read_quoted_string(FilterStateData *fstate,
+				   const char *str,
+				   PQExpBuffer pattern)
+{
+	appendPQExpBufferChar(pattern, '"');
+	str++;
+
+	while (1)
+	{
+		/*
+		 * We can ignore \r or \n chars because the string is read by
+		 * pg_get_line_buf, so these chars should be just trailing chars.
+		 */
+		if (*str == '\r' || *str == '\n')
+		{
+			str++;
+			continue;
+		}
+
+		if (*str == '\0')
+		{
+			Assert(fstate->linebuff.data);
+
+			if (!pg_get_line_buf(fstate->fp, &fstate->linebuff))
+			{
+				if (ferror(fstate->fp))
+					pg_log_error("could not read from filter file \"%s\": %m",
+								 fstate->filename);
+				else
+					pg_log_filter_error(fstate, _("unexpected end of file"));
+
+				fstate->exit_nicely(1);
+			}
+
+			str = fstate->linebuff.data;
+
+			appendPQExpBufferChar(pattern, '\n');
+			fstate->lineno++;
+		}
+
+		if (*str == '"')
+		{
+			appendPQExpBufferChar(pattern, '"');
+			str++;
+
+			if (*str == '"')
+			{
+				appendPQExpBufferChar(pattern, '"');
+				str++;
+			}
+			else
+				break;
+		}
+		else if (*str == '\\')
+		{
+			str++;
+			if (*str == 'n')
+				appendPQExpBufferChar(pattern, '\n');
+			else if (*str == '\\')
+				appendPQExpBufferChar(pattern, '\\');
+
+			str++;
+		}
+		else
+			appendPQExpBufferChar(pattern, *str++);
+	}
+
+	return str;
+}
+
+/*
+ * read_pattern - reads on object pattern from input
+ *
+ * This function will parse any valid identifier (quoted or not, qualified or
+ * not), which can also includes the full signature for routines.
+ * Note that this function takes special care to sanitize the detected
+ * identifier (removing extraneous whitespaces or other unnecessary
+ * characters).  This is necessary as most backup/restore filtering functions
+ * only recognize identifiers if they are written exactly the same way as
+ * they are output by the server.
+ *
+ * Returns a pointer to next character after the found identifier and exits
+ * on error.
+ */
+static const char *
+read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
+{
+	bool		skip_space = true;
+	bool		found_space = false;
+
+	/* Skip initial whitespace */
+	while (isspace(*str))
+		str++;
+
+	if (*str == '\0')
+	{
+		pg_log_filter_error(fstate, _("missing object name pattern"));
+		fstate->exit_nicely(1);
+	}
+
+	while (*str && *str != '#')
+	{
+		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		{
+			/*
+			 * Append space only when it is allowed, and when it was found in
+			 * original string.
+			 */
+			if (!skip_space && found_space)
+			{
+				appendPQExpBufferChar(pattern, ' ');
+				skip_space = true;
+			}
+
+			appendPQExpBufferChar(pattern, *str++);
+		}
+
+		skip_space = false;
+
+		if (*str == '"')
+		{
+			if (found_space)
+				appendPQExpBufferChar(pattern, ' ');
+
+			str = read_quoted_string(fstate, str, pattern);
+		}
+		else if (*str == ',')
+		{
+			appendPQExpBufferStr(pattern, ", ");
+			skip_space = true;
+			str++;
+		}
+		else if (*str && strchr(".()", *str))
+		{
+			appendPQExpBufferChar(pattern, *str++);
+			skip_space = true;
+		}
+
+		found_space = false;
+
+		/* skip ending whitespaces */
+		while (isspace(*str))
+		{
+			found_space = true;
+			str++;
+		}
+	}
+
+	return str;
+}
+
+/*
+ * filter_read_item - Read command/type/pattern triplet from a filter file
+ *
+ * This will parse one filter item from the filter file, and while it is a
+ * row based format a pattern may span more than one line due to how object
+ * names can be constructed.  The expected format of the filter file is:
+ *
+ * <command> <object_type> <pattern>
+ *
+ * command can be "include" or "exclude".
+ *
+ * Supported object types are described by enum FilterObjectType
+ * (see function get_object_type).
+ *
+ * pattern can be any possibly-quoted and possibly-qualified identifier.  It
+ * follows the same rules as other object include and exclude functions so it
+ * can also use wildcards.
+ *
+ * Returns true when one filter item was successfully read and parsed.  When
+ * object name contains \n chars, then more than one line from input file can
+ * be processed.  Returns false when the filter file reaches EOF. In case of
+ * error, the function will emit an appropriate error message and exit.
+ */
+bool
+filter_read_item(FilterStateData *fstate,
+				 char **objname,
+				 FilterCommandType *comtype,
+				 FilterObjectType *objtype)
+{
+	if (pg_get_line_buf(fstate->fp, &fstate->linebuff))
+	{
+		const char *str = fstate->linebuff.data;
+		const char *keyword;
+		int			size;
+		PQExpBufferData pattern;
+
+		fstate->lineno++;
+
+		/* Skip initial white spaces */
+		while (isspace(*str))
+			str++;
+
+		/*
+		 * Skip empty lines or lines where the first non-whitespace character
+		 * is a hash indicating a comment.
+		 */
+		if (*str != '\0' && *str != '#')
+		{
+			/*
+			 * First we expect sequence of two keywords, {include|exclude}
+			 * followed by the object type to operate on.
+			 */
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate,
+									_("no filter command found (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			if (is_keyword_str("include", keyword, size))
+				*comtype = FILTER_COMMAND_TYPE_INCLUDE;
+			else if (is_keyword_str("exclude", keyword, size))
+				*comtype = FILTER_COMMAND_TYPE_EXCLUDE;
+			else
+			{
+				pg_log_filter_error(fstate,
+									_("invalid filter command (expected \"include\" or \"exclude\")"));
+				fstate->exit_nicely(1);
+			}
+
+			keyword = filter_get_keyword(&str, &size);
+			if (!keyword)
+			{
+				pg_log_filter_error(fstate, _("missing filter object type"));
+				fstate->exit_nicely(1);
+			}
+
+			if (!get_object_type(keyword, size, objtype))
+			{
+				pg_log_filter_error(fstate,
+									_("unsupported filter object type: \"%.*s\""), size, keyword);
+				fstate->exit_nicely(1);
+			}
+
+			initPQExpBuffer(&pattern);
+
+			str = read_pattern(fstate, str, &pattern);
+			*objname = pattern.data;
+		}
+		else
+		{
+			*objname = NULL;
+			*comtype = FILTER_COMMAND_TYPE_NONE;
+			*objtype = FILTER_OBJECT_TYPE_NONE;
+		}
+
+		return true;
+	}
+
+	if (ferror(fstate->fp))
+	{
+		pg_log_error("could not read from filter file \"%s\": %m", fstate->filename);
+		fstate->exit_nicely(1);
+	}
+
+	return false;
+}
diff --git a/src/bin/pg_dump/filter.h b/src/bin/pg_dump/filter.h
new file mode 100644
index 0000000000..4c12f8a572
--- /dev/null
+++ b/src/bin/pg_dump/filter.h
@@ -0,0 +1,70 @@
+/*-------------------------------------------------------------------------
+ *
+ * filter.h
+ *	  Common header file for the parser of filter file
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_dump/filter.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FILTER_H
+#define FILTER_H
+
+#include "lib/stringinfo.h"
+
+/* Function signature for exit_nicely functions */
+typedef void (*exit_function) (int status);
+
+/*
+ * State data for reading filter items from stream
+ */
+typedef struct
+{
+	FILE	   *fp;
+	const char *filename;
+	exit_function exit_nicely;
+	int			lineno;
+	StringInfoData linebuff;
+} FilterStateData;
+
+/*
+ * List of command types that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_COMMAND_TYPE_NONE,
+	FILTER_COMMAND_TYPE_INCLUDE,
+	FILTER_COMMAND_TYPE_EXCLUDE,
+} FilterCommandType;
+
+/*
+ * List of objects that can be specified in filter file
+ */
+typedef enum
+{
+	FILTER_OBJECT_TYPE_NONE,
+	FILTER_OBJECT_TYPE_TABLE_DATA,
+	FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_DATABASE,
+	FILTER_OBJECT_TYPE_EXTENSION,
+	FILTER_OBJECT_TYPE_FOREIGN_DATA,
+	FILTER_OBJECT_TYPE_FUNCTION,
+	FILTER_OBJECT_TYPE_INDEX,
+	FILTER_OBJECT_TYPE_SCHEMA,
+	FILTER_OBJECT_TYPE_TABLE,
+	FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN,
+	FILTER_OBJECT_TYPE_TRIGGER,
+} FilterObjectType;
+
+extern const char *filter_object_type_name(FilterObjectType fot);
+extern void filter_init(FilterStateData *fstate, const char *filename, exit_function f_exit);
+extern void filter_free(FilterStateData *fstate);
+extern void pg_log_filter_error(FilterStateData *fstate, const char *fmt,...)
+  pg_attribute_printf(2, 3);
+extern bool filter_read_item(FilterStateData *fstate, char **objname,
+							 FilterCommandType *comtype,FilterObjectType *objtype);
+
+#endif
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index 9d59a106f3..b6603e26a5 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -7,6 +7,7 @@ pg_dump_common_sources = files(
   'compress_none.c',
   'compress_zstd.c',
   'dumputils.c',
+  'filter.c',
   'parallel.c',
   'pg_backup_archiver.c',
   'pg_backup_custom.c',
@@ -99,6 +100,7 @@ tests += {
       't/002_pg_dump.pl',
       't/003_pg_dump_with_server.pl',
       't/004_pg_dump_parallel.pl',
+      't/005_pg_dump_filterfile.pl',
       't/010_dump_connstr.pl',
     ],
   },
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 34fd0a86e9..64e2d754d1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -60,6 +60,7 @@
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "libpq/libpq-fs.h"
 #include "parallel.h"
@@ -327,6 +328,7 @@ static char *get_synchronized_snapshot(Archive *fout);
 static void setupDumpWorker(Archive *AH);
 static TableInfo *getRootTableInfo(const TableInfo *tbinfo);
 static bool forcePartitionRootLoad(const TableInfo *tbinfo);
+static void read_dump_filters(const char *filename, DumpOptions *dopt);
 
 
 int
@@ -433,6 +435,7 @@ main(int argc, char **argv)
 		{"exclude-table-and-children", required_argument, NULL, 13},
 		{"exclude-table-data-and-children", required_argument, NULL, 14},
 		{"sync-method", required_argument, NULL, 15},
+		{"filter", required_argument, NULL, 16},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -664,6 +667,10 @@ main(int argc, char **argv)
 					exit_nicely(1);
 				break;
 
+			case 16:			/* read object filters from file */
+				read_dump_filters(optarg, &dopt);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -1111,6 +1118,8 @@ help(const char *progname)
 			 "                               do NOT dump data for the specified table(s),\n"
 			 "                               including child and partition tables\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            include or exclude objects and data from dump\n"
+			 "                               based expressions in FILENAME\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --include-foreign-data=PATTERN\n"
 			 "                               include data of foreign tables on foreign\n"
@@ -18771,3 +18780,112 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
 	if (!res)
 		pg_log_warning("could not parse %s array", "reloptions");
 }
+
+/*
+ * read_dump_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_dump_filters(const char *filename, DumpOptions *dopt)
+{
+	FilterStateData fstate;
+	char	   *objname;
+	FilterCommandType comtype;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &objname, &comtype, &objtype))
+	{
+		if (comtype == FILTER_COMMAND_TYPE_INCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;		/* unreachable */
+
+				case FILTER_OBJECT_TYPE_EXTENSION:
+					simple_string_list_append(&extension_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					simple_string_list_append(&foreign_servers_include_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_include_patterns, objname);
+					dopt->include_everything = false;
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_include_patterns_and_children,
+											  objname);
+					dopt->include_everything = false;
+					break;
+			}
+		}
+		else if (comtype == FILTER_COMMAND_TYPE_EXCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+					break;
+
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+					simple_string_list_append(&tabledata_exclude_patterns,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+					simple_string_list_append(&tabledata_exclude_patterns_and_children,
+											  objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&schema_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					simple_string_list_append(&table_exclude_patterns, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+					simple_string_list_append(&table_exclude_patterns_and_children,
+											  objname);
+					break;
+			}
+		}
+		else
+		{
+			Assert(comtype == FILTER_COMMAND_TYPE_NONE);
+			Assert(objtype == FILTER_OBJECT_TYPE_NONE);
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index e2a9733d34..1b974cf7e8 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -26,6 +26,7 @@
 #include "common/string.h"
 #include "dumputils.h"
 #include "fe_utils/string_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "pg_backup.h"
 
@@ -81,6 +82,7 @@ static PGresult *executeQuery(PGconn *conn, const char *query);
 static void executeCommand(PGconn *conn, const char *query);
 static void expand_dbname_patterns(PGconn *conn, SimpleStringList *patterns,
 								   SimpleStringList *names);
+static void read_dumpall_filters(const char *filename, SimpleStringList *patterns);
 
 static char pg_dump_bin[MAXPGPATH];
 static const char *progname;
@@ -177,6 +179,7 @@ main(int argc, char *argv[])
 		{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
 		{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
 		{"rows-per-insert", required_argument, NULL, 7},
+		{"filter", required_argument, NULL, 8},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -360,6 +363,10 @@ main(int argc, char *argv[])
 				appendShellString(pgdumpopts, optarg);
 				break;
 
+			case 8:
+				read_dumpall_filters(optarg, &database_exclude_patterns);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -653,6 +660,7 @@ help(void)
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --exclude-database=PATTERN   exclude databases whose name matches PATTERN\n"));
 	printf(_("  --extra-float-digits=NUM     override default setting for extra_float_digits\n"));
+	printf(_("  --filter=FILENAME            exclude databases specified in FILENAME\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --inserts                    dump data as INSERT commands, rather than COPY\n"));
 	printf(_("  --load-via-partition-root    load partitions via the root table\n"));
@@ -1937,3 +1945,62 @@ hash_string_pointer(char *s)
 
 	return hash_bytes(ss, strlen(s));
 }
+
+/*
+ * read_dumpall_filters - retrieve database identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ *
+ * At the moment, the only allowed filter is for database exclusion.
+ */
+static void
+read_dumpall_filters(const char *filename, SimpleStringList *pattern)
+{
+	FilterStateData fstate;
+	char	   *objname;
+	FilterCommandType comtype;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit);
+
+	while (filter_read_item(&fstate, &objname, &comtype, &objtype))
+	{
+		if (comtype == FILTER_COMMAND_TYPE_INCLUDE)
+		{
+			pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+								"include",
+								filter_object_type_name(objtype));
+			exit_nicely(1);
+		}
+
+		switch (objtype)
+		{
+			case FILTER_OBJECT_TYPE_NONE:
+				break;
+			case FILTER_OBJECT_TYPE_FUNCTION:
+			case FILTER_OBJECT_TYPE_INDEX:
+			case FILTER_OBJECT_TYPE_TABLE_DATA:
+			case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+			case FILTER_OBJECT_TYPE_TRIGGER:
+			case FILTER_OBJECT_TYPE_EXTENSION:
+			case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+			case FILTER_OBJECT_TYPE_SCHEMA:
+			case FILTER_OBJECT_TYPE_TABLE:
+			case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				pg_log_filter_error(&fstate, _("unsupported filter object."));
+				exit_nicely(1);
+				break;
+
+			case FILTER_OBJECT_TYPE_DATABASE:
+				simple_string_list_append(pattern, objname);
+				break;
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..1459e02263 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -47,11 +47,13 @@
 
 #include "dumputils.h"
 #include "fe_utils/option_utils.h"
+#include "filter.h"
 #include "getopt_long.h"
 #include "parallel.h"
 #include "pg_backup_utils.h"
 
 static void usage(const char *progname);
+static void read_restore_filters(const char *filename, RestoreOptions *dopt);
 
 int
 main(int argc, char **argv)
@@ -123,6 +125,7 @@ main(int argc, char **argv)
 		{"no-publications", no_argument, &no_publications, 1},
 		{"no-security-labels", no_argument, &no_security_labels, 1},
 		{"no-subscriptions", no_argument, &no_subscriptions, 1},
+		{"filter", required_argument, NULL, 4},
 
 		{NULL, 0, NULL, 0}
 	};
@@ -286,6 +289,10 @@ main(int argc, char **argv)
 				set_dump_section(optarg, &(opts->dumpSections));
 				break;
 
+			case 4:
+				read_restore_filters(optarg, opts);
+				break;
+
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -463,6 +470,8 @@ usage(const char *progname)
 	printf(_("  -1, --single-transaction     restore as a single transaction\n"));
 	printf(_("  --disable-triggers           disable triggers during data-only restore\n"));
 	printf(_("  --enable-row-security        enable row security\n"));
+	printf(_("  --filter=FILENAME            restore or skip objects based on expressions\n"
+			 "                               in FILENAME\n"));
 	printf(_("  --if-exists                  use IF EXISTS when dropping objects\n"));
 	printf(_("  --no-comments                do not restore comments\n"));
 	printf(_("  --no-data-for-failed-tables  do not restore data of tables that could not be\n"
@@ -494,3 +503,103 @@ usage(const char *progname)
 	printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
 	printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
 }
+
+/*
+ * read_restore_filters - retrieve object identifier patterns from file
+ *
+ * Parse the specified filter file for include and exclude patterns, and add
+ * them to the relevant lists.  If the filename is "-" then filters will be
+ * read from STDIN rather than a file.
+ */
+static void
+read_restore_filters(const char *filename, RestoreOptions *opts)
+{
+	FilterStateData fstate;
+	char	   *objname;
+	FilterCommandType comtype;
+	FilterObjectType objtype;
+
+	filter_init(&fstate, filename, exit_nicely);
+
+	while (filter_read_item(&fstate, &objname, &comtype, &objtype))
+	{
+		if (comtype == FILTER_COMMAND_TYPE_INCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"include",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_FUNCTION:
+					opts->selTypes = 1;
+					opts->selFunction = 1;
+					simple_string_list_append(&opts->functionNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_INDEX:
+					opts->selTypes = 1;
+					opts->selIndex = 1;
+					simple_string_list_append(&opts->indexNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TABLE:
+					opts->selTypes = 1;
+					opts->selTable = 1;
+					simple_string_list_append(&opts->tableNames, objname);
+					break;
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					opts->selTypes = 1;
+					opts->selTrigger = 1;
+					simple_string_list_append(&opts->triggerNames, objname);
+					break;
+			}
+		}
+		else if (comtype == FILTER_COMMAND_TYPE_EXCLUDE)
+		{
+			switch (objtype)
+			{
+				case FILTER_OBJECT_TYPE_NONE:
+					break;
+				case FILTER_OBJECT_TYPE_TABLE_DATA:
+				case FILTER_OBJECT_TYPE_TABLE_DATA_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_DATABASE:
+				case FILTER_OBJECT_TYPE_EXTENSION:
+				case FILTER_OBJECT_TYPE_FOREIGN_DATA:
+				case FILTER_OBJECT_TYPE_FUNCTION:
+				case FILTER_OBJECT_TYPE_INDEX:
+				case FILTER_OBJECT_TYPE_TABLE:
+				case FILTER_OBJECT_TYPE_TABLE_AND_CHILDREN:
+				case FILTER_OBJECT_TYPE_TRIGGER:
+					pg_log_filter_error(&fstate, _("%s filter for \"%s\" is not allowed."),
+										"exclude",
+										filter_object_type_name(objtype));
+					exit_nicely(1);
+
+				case FILTER_OBJECT_TYPE_SCHEMA:
+					simple_string_list_append(&opts->schemaExcludeNames, objname);
+					break;
+			}
+		}
+		else
+		{
+			Assert(comtype == FILTER_COMMAND_TYPE_NONE);
+			Assert(objtype == FILTER_OBJECT_TYPE_NONE);
+		}
+
+		if (objname)
+			free(objname);
+	}
+
+	filter_free(&fstate);
+}
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
new file mode 100644
index 0000000000..d4c2c2340d
--- /dev/null
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -0,0 +1,776 @@
+
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $tempdir = PostgreSQL::Test::Utils::tempdir;;
+my $inputfile;
+
+my $node      = PostgreSQL::Test::Cluster->new('main');
+my $port      = $node->port;
+my $backupdir = $node->backup_dir;
+my $plainfile = "$backupdir/plain.sql";
+
+$node->init;
+$node->start;
+
+# Generate test objects
+$node->safe_psql('postgres', 'CREATE FOREIGN DATA WRAPPER dummy;');
+$node->safe_psql('postgres',
+	'CREATE SERVER dummyserver FOREIGN DATA WRAPPER dummy;');
+
+$node->safe_psql('postgres', "CREATE TABLE table_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_two(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE table_three_one(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE footab(a varchar)");
+$node->safe_psql('postgres', "CREATE TABLE bootab() inherits (footab)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"strange aaa
+name\"(a varchar)");
+$node->safe_psql(
+	'postgres', "CREATE TABLE \"
+t
+t
+\"(a int)");
+
+$node->safe_psql('postgres',
+	"INSERT INTO table_one VALUES('*** TABLE ONE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_two VALUES('*** TABLE TWO ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three VALUES('*** TABLE THREE ***')");
+$node->safe_psql('postgres',
+	"INSERT INTO table_three_one VALUES('*** TABLE THREE_ONE ***')");
+$node->safe_psql('postgres', "INSERT INTO bootab VALUES(10)");
+
+$node->safe_psql('postgres', "CREATE DATABASE sourcedb");
+$node->safe_psql('postgres', "CREATE DATABASE targetdb");
+
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo1(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo2(a int) RETURNS int AS $$ select $1 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo3(a double precision, b int) RETURNS double precision AS $$ select $1 + $2 $$ LANGUAGE sql');
+$node->safe_psql('sourcedb', 'CREATE FUNCTION foo_trg() RETURNS trigger AS $$ BEGIN RETURN NEW; END $$ LANGUAGE plpgsql');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s1');
+$node->safe_psql('sourcedb', 'CREATE SCHEMA s2');
+$node->safe_psql('sourcedb', 'CREATE TABLE s1.t1(a int)');
+$node->safe_psql('sourcedb', 'CREATE SEQUENCE s1.s1');
+$node->safe_psql('sourcedb', 'CREATE TABLE s2.t2(a int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t1(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE TABLE t2(a int, b int)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx1 ON t1(a)');
+$node->safe_psql('sourcedb', 'CREATE INDEX t1_idx2 ON t1(b)');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg1 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+$node->safe_psql('sourcedb', 'CREATE TRIGGER trg2 BEFORE INSERT ON t1 EXECUTE FUNCTION foo_trg()');
+
+#
+# Test interaction of correctly specified filter file
+#
+my ($cmd, $stdout, $stderr, $result);
+
+# Empty filterfile
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "\n # a comment and nothing more\n\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+my $dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "table one dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "table two dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "table three dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m, "table three one dumped");
+
+# Test various combinations of whitespace, comments and correct filters
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "  include   table table_one    #comment\n";
+print $inputfile "include table table_two\n";
+print $inputfile "# skip this line\n";
+print $inputfile "\n";
+print $inputfile "\t\n";
+print $inputfile "  \t# another comment\n";
+print $inputfile "exclude table_data table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter patterns as well as comments and whitespace");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump !~ qr/^CREATE TABLE public\.table_three/m, "table three not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_three_one/m,
+	"table three_one not dumped");
+ok( $dump !~ qr/^COPY public\.table_one/m,
+	"content of table one is not included");
+ok($dump =~ qr/^COPY public\.table_two/m, "content of table two is included");
+
+# Test dumping tables specified by qualified names
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table public.table_one\n";
+print $inputfile "include table \"public\".\"table_two\"\n";
+print $inputfile "include table \"public\". table_three\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m,   "dumped table one");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with exclusion of a single table");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m,   "dumped table two");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping tables with a wildcard pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_thre*\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with wildcard in pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m,   "table one not dumped");
+ok($dump !~ qr/^CREATE TABLE public\.table_two/m,   "table two not dumped");
+ok($dump =~ qr/^CREATE TABLE public\.table_three/m, "dumped table three");
+ok($dump =~ qr/^CREATE TABLE public\.table_three_one/m,
+	"dumped table three_one");
+
+# Test dumping table with multiline quoted tablename
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"strange aaa
+name\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with multiline names requiring quoting");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding multiline quoted tablename from dump
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table \"strange aaa\\nname\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public.\"strange aaa/m,
+	"dump table with new line in name");
+
+# Test excluding an entire schema
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema public\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"exclude the public schema");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test including and excluding an entire schema by multiple filterfiles
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema public\n";
+close $inputfile;
+
+open my $alt_inputfile, '>', "$tempdir/inputfile2.txt"
+  or die "unable to open filterfile for writing";
+print $alt_inputfile "exclude schema public\n";
+close $alt_inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"--filter=$tempdir/inputfile2.txt", 'postgres'
+	],
+	"exclude the public schema with multiple filters");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE/m, "no table dumped");
+
+# Test dumping a table with a single leading newline on a row
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"
+t
+t
+\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table \"\\nt\\nt\\n\"";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public.\"\nt\nt\n\" \($/ms,
+	"dump table with multiline strange name");
+
+#########################################
+# Test foreign_data
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include foreign_data doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching foreign servers were found for pattern/,
+	"dump nonexisting foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile, "include foreign_data dummyserver\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"dump foreign_data with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE SERVER dummyserver/m, "dump foreign server");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude foreign_data dummy*\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/exclude filter for "foreign data" is not allowed/,
+	"erroneously exclude foreign server");
+
+#########################################
+# Test broken input format
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: invalid object type specified, should be table, schema, foreign_data or data"
+);
+
+# Test missing object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/missing object name/,
+	"invalid syntax: missing object identifier pattern");
+
+# Test adding extra content after the object identifier pattern
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table one";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/no matching tables were found/,
+	"invalid syntax: extra content after object identifier pattern");
+
+#########################################
+# Combined with --strict-names
+
+# First ensure that a matching filter works
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_one\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	"strict names with matching pattern");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_one/m, "no table dumped");
+
+# Now append a pattern to the filter file which doesn't resolve
+open $inputfile, '>>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_nonexisting_name";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		'--strict-names', 'postgres'
+	],
+	qr/no matching tables were found/,
+	"inclusion of non-existing objects with --strict names");
+
+#########################################
+# pg_dumpall tests
+
+###########################
+# Test dumping all tables except one
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude database postgres\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	"dump tables with exclusion of a database");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^\\connect postgres/m, "database postgres is not dumped");
+ok($dump =~ qr/^\\connect template1/m, "database template1 is dumped");
+
+# Test invalid filter command
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "k";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/invalid filter command/,
+	"invalid syntax: incorrect filter command");
+
+# Test invalid object type
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/unsupported filter object type: "xxx"/,
+	"invalid syntax: exclusion of non-existing object type"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table foo";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dumpall', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/pg_dumpall: error: invalid format in filter/,
+	"invalid syntax: exclusion of unsupported object type"
+);
+
+#########################################
+# pg_restore tests
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'postgres'
+	],
+	"dump all tables");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table table_two";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore tables with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.table_two/m, "wanted table restored");
+ok($dump !~ qr/^CREATE TABLE public\.table_one/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include table_data xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/include filter for "table data" is not allowed/,
+	"invalid syntax: inclusion of unallowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/include filter for "extension" is not allowed/,
+	"invalid syntax: inclusion of unallowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude extension xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/exclude filter for "extension" is not allowed/,
+	"invalid syntax: exclusion of unallowed object"
+);
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude table_data xxx";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt"
+	],
+	qr/exclude filter for "table data" is not allowed/,
+	"invalid syntax: exclusion of unallowed object"
+);
+
+#########################################
+# test restore of other objects
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', "$tempdir/filter_test.dump",
+		"-Fc", 'sourcedb'
+	],
+	"dump all objects from sourcedb");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function foo1(integer)";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo1/m, "wanted function restored");
+ok($dump !~ qr/^CREATE TABLE public\.foo2/m, "unwanted function is not restored");
+
+# this should be white space tolerant (against the -P argument)
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include function  foo3 ( double  precision ,   integer)  ";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE FUNCTION public\.foo3/m, "wanted function restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include index t1_idx1\n";
+
+# attention! this hit pg_restore bug - correct name of trigger is "trg1"
+# not "t1 trg1". Should be fixed when pg_restore will be fixed
+print $inputfile "include trigger t1 trg1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE INDEX t1_idx1/m, "wanted index restored");
+ok($dump !~ qr/^CREATE INDEX t2_idx2/m, "unwanted index are not restored");
+ok($dump =~ qr/^CREATE TRIGGER trg1/m, "wanted trigger restored");
+ok($dump !~ qr/^CREATE TRIGGER trg2/m, "unwanted trigger is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE s1\.t1/m, "wanted table from schema restored");
+ok($dump =~ qr/^CREATE SEQUENCE s1\.s1/m, "wanted sequence from schema restored");
+ok($dump !~ qr/^CREATE TABLE s2\t2/m, "unwanted table is not restored");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "exclude schema s1\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_restore', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt",
+		"-Fc", "$tempdir/filter_test.dump"
+	],
+	"restore function with filter");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE s1\.t1/m, "unwanted table from schema is not restored");
+ok($dump !~ qr/^CREATE SEQUENCE s1\.s1/m, "unwanted sequence from schema is not restored");
+ok($dump =~ qr/^CREATE TABLE s2\.t2/m, "wanted table restored");
+ok($dump =~ qr/^CREATE TABLE public\.t1/m, "wanted table restored");
+
+#########################################
+# test of supported syntax
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "include table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump !~ qr/^CREATE TABLE public\.bootab/m,   "exclude dumped children table");
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+
+print $inputfile "exclude table_data_and_children footab\n";
+close $inputfile;
+
+command_ok(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	"filter file without patterns");
+
+$dump = slurp_file($plainfile);
+
+ok($dump =~ qr/^CREATE TABLE public\.bootab/m,   "dumped children table");
+ok($dump !~ qr/^COPY public\.bootab/m,   "exclude dumped children table");
+
+#########################################
+# Test extension
+
+open $inputfile, '>', "$tempdir/inputfile.txt"
+  or die "unable to open filterfile for writing";
+print $inputfile "include extension doesnt_exists\n";
+close $inputfile;
+
+command_fails_like(
+	[
+		'pg_dump', '-p', $port, '-f', $plainfile,
+		"--filter=$tempdir/inputfile.txt", 'postgres'
+	],
+	qr/pg_dump: error: no matching extensions were found/,
+	"dump nonexisting extension");
+
+
+done_testing();
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 84f648c174..bcbcd8116f 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -455,6 +455,7 @@ sub mkvcbuild
 	$pgdumpall->AddIncludeDir('src/backend');
 	$pgdumpall->AddFile('src/bin/pg_dump/pg_dumpall.c');
 	$pgdumpall->AddFile('src/bin/pg_dump/dumputils.c');
+	$pgdumpall->AddFile('src/bin/pg_dump/filter.c');
 	$pgdumpall->AddLibrary('ws2_32.lib');
 
 	my $pgrestore = AddSimpleFrontend('pg_dump', 1);
-- 
2.32.1 (Apple Git-133)

#201Erik Rijkers
er@xs4all.nl
In reply to: Daniel Gustafsson (#200)
Re: proposal: possibility to read dumped table's name from file

Op 11/21/23 om 22:10 schreef Daniel Gustafsson:

On 20 Nov 2023, at 06:20, Pavel Stehule <pavel.stehule@gmail.com> wrote:

The attached is pretty close to a committable patch IMO, review is welcome on
both the patch and commit message. I tried to identify all reviewers over the
past 3+ years but I might have missed someone.

I've tested this, albeit mostly in the initial iterations (*shrug* but
a mention is nice)

Erik Rijkers

Show quoted text

--
Daniel Gustafsson

#202Daniel Gustafsson
daniel@yesql.se
In reply to: Erik Rijkers (#201)
Re: proposal: possibility to read dumped table's name from file

On 22 Nov 2023, at 05:27, Erik Rijkers <er@xs4all.nl> wrote:

Op 11/21/23 om 22:10 schreef Daniel Gustafsson:

On 20 Nov 2023, at 06:20, Pavel Stehule <pavel.stehule@gmail.com> wrote:

The attached is pretty close to a committable patch IMO, review is welcome on
both the patch and commit message. I tried to identify all reviewers over the
past 3+ years but I might have missed someone.

I took another look at this, found some more polish that was needed, added
another testcase and ended up pushing it.

I've tested this, albeit mostly in the initial iterations (*shrug* but a mention is nice)

As I mentioned above it's easy to miss when reviewing three years worth of
emails, no-one was intentionally left out. I went back and looked and added
you as a reviewer. Thanks for letting me know.

--
Daniel Gustafsson

#203Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#202)
Re: proposal: possibility to read dumped table's name from file

Hi

st 29. 11. 2023 v 15:44 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 22 Nov 2023, at 05:27, Erik Rijkers <er@xs4all.nl> wrote:

Op 11/21/23 om 22:10 schreef Daniel Gustafsson:

On 20 Nov 2023, at 06:20, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

The attached is pretty close to a committable patch IMO, review is

welcome on

both the patch and commit message. I tried to identify all reviewers

over the

past 3+ years but I might have missed someone.

I took another look at this, found some more polish that was needed, added
another testcase and ended up pushing it.

I've tested this, albeit mostly in the initial iterations (*shrug* but

a mention is nice)

As I mentioned above it's easy to miss when reviewing three years worth of
emails, no-one was intentionally left out. I went back and looked and
added
you as a reviewer. Thanks for letting me know.

Thank you very much

Regards

Pavel

Show quoted text

--
Daniel Gustafsson

#204Tom Lane
tgl@sss.pgh.pa.us
In reply to: Daniel Gustafsson (#202)
Re: proposal: possibility to read dumped table's name from file

Daniel Gustafsson <daniel@yesql.se> writes:

I took another look at this, found some more polish that was needed, added
another testcase and ended up pushing it.

mamba is unhappy because this uses <ctype.h> functions without
casting their arguments to unsigned char:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&amp;dt=2023-11-30%2002%3A53%3A25

(I had not realized that we still had buildfarm animals that would
complain about this ... but I'm glad we do, because it's a hazard.
POSIX is quite clear that the behavior is undefined for signed chars.)

regards, tom lane

#205Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#204)
1 attachment(s)
Re: proposal: possibility to read dumped table's name from file

Hi

čt 30. 11. 2023 v 4:40 odesílatel Tom Lane <tgl@sss.pgh.pa.us> napsal:

Daniel Gustafsson <daniel@yesql.se> writes:

I took another look at this, found some more polish that was needed,

added

another testcase and ended up pushing it.

mamba is unhappy because this uses <ctype.h> functions without
casting their arguments to unsigned char:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&amp;dt=2023-11-30%2002%3A53%3A25

(I had not realized that we still had buildfarm animals that would
complain about this ... but I'm glad we do, because it's a hazard.
POSIX is quite clear that the behavior is undefined for signed chars.)

here is a patch

Regards

Pavel

Show quoted text

regards, tom lane

Attachments:

fix_warning_array_subscript_has_type_char.patchtext/x-patch; charset=US-ASCII; name=fix_warning_array_subscript_has_type_char.patchDownload
commit 29a570f6b003513ef93691d92e01ea80fab11844
Author: okbob@github.com <pavel.stehule@gmail.com>
Date:   Thu Nov 30 07:10:58 2023 +0100

    fix warning array subscript has type 'char'

diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index dff871c7cd..d79b6da27c 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -185,14 +185,14 @@ filter_get_keyword(const char **line, int *size)
 	*size = 0;
 
 	/* Skip initial whitespace */
-	while (isspace(*ptr))
+	while (isspace((unsigned char) *ptr))
 		ptr++;
 
-	if (isalpha(*ptr))
+	if (isalpha((unsigned char) *ptr))
 	{
 		result = ptr++;
 
-		while (isalpha(*ptr) || *ptr == '_')
+		while (isalpha((unsigned char) *ptr) || *ptr == '_')
 			ptr++;
 
 		*size = ptr - result;
@@ -301,7 +301,7 @@ read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
 	bool		found_space = false;
 
 	/* Skip initial whitespace */
-	while (isspace(*str))
+	while (isspace((unsigned char) *str))
 		str++;
 
 	if (*str == '\0')
@@ -312,7 +312,7 @@ read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
 
 	while (*str && *str != '#')
 	{
-		while (*str && !isspace(*str) && !strchr("#,.()\"", *str))
+		while (*str && !isspace((unsigned char) *str) && !strchr("#,.()\"", *str))
 		{
 			/*
 			 * Append space only when it is allowed, and when it was found in
@@ -351,7 +351,7 @@ read_pattern(FilterStateData *fstate, const char *str, PQExpBuffer pattern)
 		found_space = false;
 
 		/* skip ending whitespaces */
-		while (isspace(*str))
+		while (isspace((unsigned char) *str))
 		{
 			found_space = true;
 			str++;
@@ -400,7 +400,7 @@ filter_read_item(FilterStateData *fstate,
 		fstate->lineno++;
 
 		/* Skip initial white spaces */
-		while (isspace(*str))
+		while (isspace((unsigned char) *str))
 			str++;
 
 		/*
#206Daniel Gustafsson
daniel@yesql.se
In reply to: Pavel Stehule (#205)
Re: proposal: possibility to read dumped table's name from file

On 30 Nov 2023, at 07:13, Pavel Stehule <pavel.stehule@gmail.com> wrote:
čt 30. 11. 2023 v 4:40 odesílatel Tom Lane <tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> napsal:
Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se>> writes:

I took another look at this, found some more polish that was needed, added
another testcase and ended up pushing it.

mamba is unhappy because this uses <ctype.h> functions without
casting their arguments to unsigned char:

Thanks for the heads-up.

here is a patch

I agree with this fix, and have applied it.

--
Daniel Gustafsson

#207Pavel Stehule
pavel.stehule@gmail.com
In reply to: Daniel Gustafsson (#206)
Re: proposal: possibility to read dumped table's name from file

čt 30. 11. 2023 v 14:05 odesílatel Daniel Gustafsson <daniel@yesql.se>
napsal:

On 30 Nov 2023, at 07:13, Pavel Stehule <pavel.stehule@gmail.com> wrote:
čt 30. 11. 2023 v 4:40 odesílatel Tom Lane <tgl@sss.pgh.pa.us <mailto:

tgl@sss.pgh.pa.us>> napsal:

Daniel Gustafsson <daniel@yesql.se <mailto:daniel@yesql.se>> writes:

I took another look at this, found some more polish that was needed,

added

another testcase and ended up pushing it.

mamba is unhappy because this uses <ctype.h> functions without
casting their arguments to unsigned char:

Thanks for the heads-up.

here is a patch

I agree with this fix, and have applied it.

Thank you

Pavel

Show quoted text

--
Daniel Gustafsson