sequences and pg_upgrade
I was toying with a couple of ideas that would involve changing the
storage of sequences. (Say, for the sake of discussion, removing the
problematic/useless sequence_name field.) This would cause problems for
pg_upgrade, because pg_upgrade copies the "heap" storage of sequences
like it does for normal tables, and we have no facilities for effecting
any changes during that.
There was a previous discussion in the early days of pg_migrator, which
resulted in the current behavior:
/messages/by-id/20090713220112.GF7933@klana.box
This also alluded to what I think was the last change in the sequence
storage format (10a3471bed7b57fb986a5be8afdee5f0dda419de) between
versions 8.3 and 8.4. How did pg_upgrade handle that?
I think the other solution mentioned in that thread would also work:
Have pg_upgrade treat sequences more like system catalogs, whose format
changes between major releases, and transferred them via the
dump/restore route. So instead of copying the disk files, issue a
setval call, and the sequence should be all set up.
Am I missing anything?
Attached is a rough patch set that would implement that.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
0001-pg_dump-Separate-table-data-and-sequence-data-object.patchtext/x-patch; name=0001-pg_dump-Separate-table-data-and-sequence-data-object.patchDownload
From 0c8f9bb630f48e83dc4dbe36e742db8e20f6b523 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH 1/3] pg_dump: Separate table data and sequence data object
types
---
src/bin/pg_dump/pg_dump.c | 11 +++++++----
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/pg_dump_sort.c | 7 +++++++
3 files changed, 15 insertions(+), 4 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a5c2d09..160bc41 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -2133,6 +2133,8 @@ makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo, bool oids)
if (tbinfo->relkind == RELKIND_MATVIEW)
tdinfo->dobj.objType = DO_REFRESH_MATVIEW;
+ else if (tbinfo->relkind == RELKIND_SEQUENCE)
+ tdinfo->dobj.objType = DO_SEQUENCE_DATA;
else
tdinfo->dobj.objType = DO_TABLE_DATA;
@@ -9382,11 +9384,11 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_TRANSFORM:
dumpTransform(fout, (TransformInfo *) dobj);
break;
+ case DO_SEQUENCE_DATA:
+ dumpSequenceData(fout, (TableDataInfo *) dobj);
+ break;
case DO_TABLE_DATA:
- if (((TableDataInfo *) dobj)->tdtable->relkind == RELKIND_SEQUENCE)
- dumpSequenceData(fout, (TableDataInfo *) dobj);
- else
- dumpTableData(fout, (TableDataInfo *) dobj);
+ dumpTableData(fout, (TableDataInfo *) dobj);
break;
case DO_DUMMY_TYPE:
/* table rowtypes and array types are never dumped separately */
@@ -17482,6 +17484,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
addObjectDependency(preDataBound, dobj->dumpId);
break;
case DO_TABLE_DATA:
+ case DO_SEQUENCE_DATA:
case DO_BLOB_DATA:
/* Data objects: must come between the boundaries */
addObjectDependency(dobj, preDataBound->dumpId);
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 2bfa2d9..6cc78d1 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -63,6 +63,7 @@ typedef enum
DO_PROCLANG,
DO_CAST,
DO_TABLE_DATA,
+ DO_SEQUENCE_DATA,
DO_DUMMY_TYPE,
DO_TSPARSER,
DO_TSDICT,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index d87f08d..9ca3d2b 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -60,6 +60,7 @@ static const int oldObjectTypePriority[] =
2, /* DO_PROCLANG */
2, /* DO_CAST */
11, /* DO_TABLE_DATA */
+ 11, /* DO_SEQUENCE_DATA */
7, /* DO_DUMMY_TYPE */
4, /* DO_TSPARSER */
4, /* DO_TSDICT */
@@ -111,6 +112,7 @@ static const int newObjectTypePriority[] =
2, /* DO_PROCLANG */
10, /* DO_CAST */
23, /* DO_TABLE_DATA */
+ 23, /* DO_SEQUENCE_DATA */
19, /* DO_DUMMY_TYPE */
12, /* DO_TSPARSER */
14, /* DO_TSDICT */
@@ -1433,6 +1435,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"TABLE DATA %s (ID %d OID %u)",
obj->name, obj->dumpId, obj->catId.oid);
return;
+ case DO_SEQUENCE_DATA:
+ snprintf(buf, bufsize,
+ "SEQUENCE DATA %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
case DO_DUMMY_TYPE:
snprintf(buf, bufsize,
"DUMMY TYPE %s (ID %d OID %u)",
--
2.9.3
0002-pg_dump-Add-sequence-data-option.patchtext/x-patch; name=0002-pg_dump-Add-sequence-data-option.patchDownload
From 26325789ef3cb0e898d94f06b395ae4c64e3b2e9 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH 2/3] pg_dump: Add --sequence-data option
---
src/bin/pg_dump/pg_backup.h | 2 ++
src/bin/pg_dump/pg_backup_archiver.c | 6 +++++-
src/bin/pg_dump/pg_dump.c | 22 ++++++++++++++++++++++
3 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 4afa92f..2fdd364 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -117,6 +117,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
+ int sequence_data;
} RestoreOptions;
typedef struct _dumpOptions
@@ -150,6 +151,7 @@ typedef struct _dumpOptions
int outputNoTablespaces;
int use_setsessauth;
int enable_row_security;
+ int sequence_data;
/* default, if no "inclusion" switches appear, is to dump everything */
bool include_everything;
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 05bdbdb..d8bee2e 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -166,6 +166,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->lockWaitTimeout = ropt->lockWaitTimeout;
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
+ dopt->sequence_data = ropt->sequence_data;
return dopt;
}
@@ -2826,7 +2827,10 @@ _tocEntryRequired(TocEntry *te, teSection curSection, RestoreOptions *ropt)
/* Mask it if we only want schema */
if (ropt->schemaOnly)
- res = res & REQ_SCHEMA;
+ {
+ if (!(ropt->sequence_data && strcmp(te->desc, "SEQUENCE SET") == 0))
+ res = res & REQ_SCHEMA;
+ }
/* Mask it if we only want data */
if (ropt->dataOnly)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 160bc41..7ff957a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -220,6 +220,7 @@ static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids);
+static void getSequenceData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo, bool oids);
static void buildMatViewRefreshDependencies(Archive *fout);
static void getTableDataFKConstraints(void);
@@ -342,6 +343,7 @@ main(int argc, char **argv)
{"quote-all-identifiers", no_argument, "e_all_identifiers, 1},
{"role", required_argument, NULL, 3},
{"section", required_argument, NULL, 5},
+ {"sequence-data", no_argument, &dopt.sequence_data, 1},
{"serializable-deferrable", no_argument, &dopt.serializable_deferrable, 1},
{"snapshot", required_argument, NULL, 6},
{"strict-names", no_argument, &strict_names, 1},
@@ -752,6 +754,9 @@ main(int argc, char **argv)
getTableDataFKConstraints();
}
+ if (dopt.schemaOnly && dopt.sequence_data)
+ getSequenceData(&dopt, tblinfo, numTables, dopt.oids);
+
if (dopt.outputBlobs)
getBlobs(fout);
@@ -835,6 +840,7 @@ main(int argc, char **argv)
ropt->lockWaitTimeout = dopt.lockWaitTimeout;
ropt->include_everything = dopt.include_everything;
ropt->enable_row_security = dopt.enable_row_security;
+ ropt->sequence_data = dopt.sequence_data;
if (compressLevel == -1)
ropt->compression = 0;
@@ -2094,6 +2100,22 @@ getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids)
}
/*
+ * getSequenceData -
+ * set up dumpable objects representing the contents of sequences
+ */
+static void
+getSequenceData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids)
+{
+ int i;
+
+ for (i = 0; i < numTables; i++)
+ {
+ if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA && tblinfo[i].relkind == RELKIND_SEQUENCE)
+ makeTableDataInfo(dopt, &(tblinfo[i]), oids);
+ }
+}
+
+/*
* Make a dumpable object for the data of this specific table
*
* Note: we make a TableDataInfo if and only if we are going to dump the
--
2.9.3
0003-pg_upgrade-Skip-copying-sequence-files.patchtext/x-patch; name=0003-pg_upgrade-Skip-copying-sequence-files.patchDownload
From 6a1f0500dee915e791eb36db502ff6e6bc5ca750 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH 3/3] pg_upgrade: Skip copying sequence files
Have pg_dump/pg_restore do it.
---
src/bin/pg_upgrade/dump.c | 2 +-
src/bin/pg_upgrade/info.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 81fb725..97f7d66 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -62,7 +62,7 @@ generate_old_dump(void)
snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
parallel_exec_prog(log_file_name, NULL,
- "\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
+ "\"%s/pg_dump\" %s --schema-only --sequence-data --quote-all-identifiers "
"--binary-upgrade --format=custom %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 1200c7f..6ea5720 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -444,7 +444,7 @@ get_rel_infos(ClusterInfo *cluster, DbInfo *dbinfo)
" SELECT c.oid, 0::oid, 0::oid "
" FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n "
" ON c.relnamespace = n.oid "
- " WHERE relkind IN ('r', 'm', 'S') AND "
+ " WHERE relkind IN ('r', 'm') AND "
/* exclude possible orphaned temp tables */
" ((n.nspname !~ '^pg_temp_' AND "
" n.nspname !~ '^pg_toast_temp_' AND "
--
2.9.3
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
I was toying with a couple of ideas that would involve changing the
storage of sequences. (Say, for the sake of discussion, removing the
problematic/useless sequence_name field.) This would cause problems for
pg_upgrade, because pg_upgrade copies the "heap" storage of sequences
like it does for normal tables, and we have no facilities for effecting
any changes during that.
There was a previous discussion in the early days of pg_migrator, which
resulted in the current behavior:
/messages/by-id/20090713220112.GF7933@klana.box
This also alluded to what I think was the last change in the sequence
storage format (10a3471bed7b57fb986a5be8afdee5f0dda419de) between
versions 8.3 and 8.4. How did pg_upgrade handle that?
I think it probably never did handle that. pg_upgrade doesn't currently
claim to support migrating from 8.3, and the thread you mention shows that
the original attempt at 8.3->8.4 migration crashed-and-burned for numerous
unrelated reasons. We may not have ever got to the point of noticing that
10a3471be also created a problem.
I think the other solution mentioned in that thread would also work:
Have pg_upgrade treat sequences more like system catalogs, whose format
changes between major releases, and transferred them via the
dump/restore route. So instead of copying the disk files, issue a
setval call, and the sequence should be all set up.
Seems reasonable.
If you're proposing to expose --sequence-data as a user-visible option,
the patch set lacks documentation. But I wonder whether it shouldn't
simply be a side-effect of --binary-upgrade. It seems a tad
non-orthogonal for a user switch.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Aug 30, 2016 at 08:46:48AM -0400, Peter Eisentraut wrote:
I think the other solution mentioned in that thread would also work:
Have pg_upgrade treat sequences more like system catalogs, whose format
changes between major releases, and transferred them via the
dump/restore route. So instead of copying the disk files, issue a
setval call, and the sequence should be all set up.Am I missing anything?
Looks straight-forward to me.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-08-30 08:46:48 -0400, Peter Eisentraut wrote:
I was toying with a couple of ideas that would involve changing the
storage of sequences. (Say, for the sake of discussion, removing the
problematic/useless sequence_name field.)
I'd be quite interested to know what changes that are...
I think the other solution mentioned in that thread would also work:
Have pg_upgrade treat sequences more like system catalogs, whose format
changes between major releases, and transferred them via the
dump/restore route. So instead of copying the disk files, issue a
setval call, and the sequence should be all set up.
+1.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: tested, passed
Documentation: tested, failed
Thank you for the patch.
As I see there are no objections in the discussion, all the patches look clear.
Could you clarify, please, why do we dump sequence in schemaOnly mode?
+ if (dopt.schemaOnly && dopt.sequence_data)
+ getSequenceData(&dopt, tblinfo, numTables, dopt.oids);
Example:
postgres=# create table t(i serial, data text);
postgres=# insert into t(data) values ('aaa');
pg_dump -d postgres --sequence-data --schema-only > ../reviews/dump_pg
Then restore it into newdb and add new value.
newdb=# insert into t(data) values ('aaa');
INSERT 0 1
newdb=# select * from t;
i | data
---+------
2 | aaa
I'm not an experienced user, but I thought that while doing dump/restore
of schema of database we reset all the data. Why should the table in newly
created (via pg_restore) database have non-default sequence value?
I also did some other tests and all of them were passed.
One more thing to do is a documentation for the new option.
You should update help() function in pg_dump.c and also add some
notes to pg_dump.sgml and probably to pgupgrade.sgml.
The new status of this patch is: Waiting on Author
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 9/14/16 8:52 AM, Anastasia Lubennikova wrote:
Could you clarify, please, why do we dump sequence in schemaOnly mode? + if (dopt.schemaOnly && dopt.sequence_data) + getSequenceData(&dopt, tblinfo, numTables, dopt.oids);
The point of this patch is that with the new option, you can dump
sequence data (but not table data) alongside with the schema. This
would be used by pg_upgrade for the reasons described at the beginning
of the thread.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
15.09.2016 15:29, Peter Eisentraut:
On 9/14/16 8:52 AM, Anastasia Lubennikova wrote:
Could you clarify, please, why do we dump sequence in schemaOnly mode? + if (dopt.schemaOnly && dopt.sequence_data) + getSequenceData(&dopt, tblinfo, numTables, dopt.oids);The point of this patch is that with the new option, you can dump
sequence data (but not table data) alongside with the schema. This
would be used by pg_upgrade for the reasons described at the beginning
of the thread.
Oh, thank you. Now I see.
Somewhy I thought that it *always* dumps sequence data in schemaOnly mode.
Which is definitely not true.
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Here is an updated patch set. Compared to the initial set, I have
changed pg_dump's sorting priorities so that sequence data is always
after table data. This would otherwise have introduced a problem
because sortDataAndIndexObjectsBySize() only considers consecutive
DO_TABLE_DATA entries. Also, I have removed the separate
--sequence-data switch from pg_dump and made it implicit in
--binary-upgrade. (So the previous patches 0002 and 0003 have been
combined, because it's no longer a separate feature.)
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
v2-0001-pg_dump-Separate-table-and-sequence-data-object-t.patchinvalid/octet-stream; name=v2-0001-pg_dump-Separate-table-and-sequence-data-object-t.patchDownload
From 5a676b0e0ecdda04facb3fd4327e6b9615f6c8e4 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH v2 1/2] pg_dump: Separate table and sequence data object types
Instead of handling both sequence data and table data internally as
"table data", handle sequences separately under a "sequence set" type.
We already handled materialized view data differently, so it makes the
code somewhat cleaner to handle each relation kind separately at the top
level.
This does not change the output format, since there already was a
separate "SEQUENCE SET" archive entry type. A noticeable difference is
that SEQUENCE SET entries now always appear after TABLE DATA entries.
And in parallel mode there is less sorting to do, because the sequence
data entries are no longer considered table data.
---
src/bin/pg_dump/pg_dump.c | 11 +++++----
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/pg_dump_sort.c | 51 ++++++++++++++++++++++++------------------
3 files changed, 37 insertions(+), 26 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ba9c276..004499c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -2141,6 +2141,8 @@ makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo, bool oids)
if (tbinfo->relkind == RELKIND_MATVIEW)
tdinfo->dobj.objType = DO_REFRESH_MATVIEW;
+ else if (tbinfo->relkind == RELKIND_SEQUENCE)
+ tdinfo->dobj.objType = DO_SEQUENCE_SET;
else
tdinfo->dobj.objType = DO_TABLE_DATA;
@@ -9391,11 +9393,11 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_TRANSFORM:
dumpTransform(fout, (TransformInfo *) dobj);
break;
+ case DO_SEQUENCE_SET:
+ dumpSequenceData(fout, (TableDataInfo *) dobj);
+ break;
case DO_TABLE_DATA:
- if (((TableDataInfo *) dobj)->tdtable->relkind == RELKIND_SEQUENCE)
- dumpSequenceData(fout, (TableDataInfo *) dobj);
- else
- dumpTableData(fout, (TableDataInfo *) dobj);
+ dumpTableData(fout, (TableDataInfo *) dobj);
break;
case DO_DUMMY_TYPE:
/* table rowtypes and array types are never dumped separately */
@@ -17491,6 +17493,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
addObjectDependency(preDataBound, dobj->dumpId);
break;
case DO_TABLE_DATA:
+ case DO_SEQUENCE_SET:
case DO_BLOB_DATA:
/* Data objects: must come between the boundaries */
addObjectDependency(dobj, preDataBound->dumpId);
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 2bfa2d9..a188410 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -63,6 +63,7 @@ typedef enum
DO_PROCLANG,
DO_CAST,
DO_TABLE_DATA,
+ DO_SEQUENCE_SET,
DO_DUMMY_TYPE,
DO_TSPARSER,
DO_TSDICT,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index d87f08d..8672e06 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -52,14 +52,15 @@ static const int oldObjectTypePriority[] =
5, /* DO_CONVERSION */
6, /* DO_TABLE */
8, /* DO_ATTRDEF */
- 15, /* DO_INDEX */
- 16, /* DO_RULE */
- 17, /* DO_TRIGGER */
- 14, /* DO_CONSTRAINT */
- 18, /* DO_FK_CONSTRAINT */
+ 16, /* DO_INDEX */
+ 17, /* DO_RULE */
+ 18, /* DO_TRIGGER */
+ 15, /* DO_CONSTRAINT */
+ 19, /* DO_FK_CONSTRAINT */
2, /* DO_PROCLANG */
2, /* DO_CAST */
11, /* DO_TABLE_DATA */
+ 12, /* DO_SEQUENCE_SET */
7, /* DO_DUMMY_TYPE */
4, /* DO_TSPARSER */
4, /* DO_TSDICT */
@@ -67,15 +68,15 @@ static const int oldObjectTypePriority[] =
4, /* DO_TSCONFIG */
4, /* DO_FDW */
4, /* DO_FOREIGN_SERVER */
- 19, /* DO_DEFAULT_ACL */
+ 20, /* DO_DEFAULT_ACL */
4, /* DO_TRANSFORM */
9, /* DO_BLOB */
- 12, /* DO_BLOB_DATA */
+ 13, /* DO_BLOB_DATA */
10, /* DO_PRE_DATA_BOUNDARY */
- 13, /* DO_POST_DATA_BOUNDARY */
- 20, /* DO_EVENT_TRIGGER */
- 15, /* DO_REFRESH_MATVIEW */
- 21 /* DO_POLICY */
+ 14, /* DO_POST_DATA_BOUNDARY */
+ 21, /* DO_EVENT_TRIGGER */
+ 16, /* DO_REFRESH_MATVIEW */
+ 22 /* DO_POLICY */
};
/*
@@ -103,14 +104,15 @@ static const int newObjectTypePriority[] =
11, /* DO_CONVERSION */
18, /* DO_TABLE */
20, /* DO_ATTRDEF */
- 27, /* DO_INDEX */
- 28, /* DO_RULE */
- 29, /* DO_TRIGGER */
- 26, /* DO_CONSTRAINT */
- 30, /* DO_FK_CONSTRAINT */
+ 28, /* DO_INDEX */
+ 29, /* DO_RULE */
+ 30, /* DO_TRIGGER */
+ 27, /* DO_CONSTRAINT */
+ 31, /* DO_FK_CONSTRAINT */
2, /* DO_PROCLANG */
10, /* DO_CAST */
23, /* DO_TABLE_DATA */
+ 24, /* DO_SEQUENCE_SET */
19, /* DO_DUMMY_TYPE */
12, /* DO_TSPARSER */
14, /* DO_TSDICT */
@@ -118,15 +120,15 @@ static const int newObjectTypePriority[] =
15, /* DO_TSCONFIG */
16, /* DO_FDW */
17, /* DO_FOREIGN_SERVER */
- 31, /* DO_DEFAULT_ACL */
+ 32, /* DO_DEFAULT_ACL */
3, /* DO_TRANSFORM */
21, /* DO_BLOB */
- 24, /* DO_BLOB_DATA */
+ 25, /* DO_BLOB_DATA */
22, /* DO_PRE_DATA_BOUNDARY */
- 25, /* DO_POST_DATA_BOUNDARY */
- 32, /* DO_EVENT_TRIGGER */
- 33, /* DO_REFRESH_MATVIEW */
- 34 /* DO_POLICY */
+ 26, /* DO_POST_DATA_BOUNDARY */
+ 33, /* DO_EVENT_TRIGGER */
+ 34, /* DO_REFRESH_MATVIEW */
+ 35 /* DO_POLICY */
};
static DumpId preDataBoundId;
@@ -1433,6 +1435,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"TABLE DATA %s (ID %d OID %u)",
obj->name, obj->dumpId, obj->catId.oid);
return;
+ case DO_SEQUENCE_SET:
+ snprintf(buf, bufsize,
+ "SEQUENCE SET %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
case DO_DUMMY_TYPE:
snprintf(buf, bufsize,
"DUMMY TYPE %s (ID %d OID %u)",
--
2.10.0
v2-0002-pg_upgrade-Upgrade-sequence-data-via-pg_dump.patchinvalid/octet-stream; name=v2-0002-pg_upgrade-Upgrade-sequence-data-via-pg_dump.patchDownload
From 746582b0d3fe85bb4f8da62245e55794a96208f2 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH v2 2/2] pg_upgrade: Upgrade sequence data via pg_dump
Previously, pg_upgrade migrated sequence data like tables by copying the
on-disk file. This does not allow any changes in the on-disk format for
sequences. It's simpler to just have pg_dump set the new sequence
values as it normally does. To do that, create a hidden submode in
pg_dump that dumps sequence data even when a schema-only dump is
requested, and trigger that submode in binary upgrade mode. (This new
submode could easily be exposed as a command-line option, but it has
limited use outside of pg_dump and would probably cause some confusion,
so we don't do that at this time.)
---
src/bin/pg_dump/pg_backup.h | 3 +++
src/bin/pg_dump/pg_backup_archiver.c | 6 +++++-
src/bin/pg_dump/pg_dump.c | 19 +++++++++++++++----
src/bin/pg_upgrade/info.c | 2 +-
4 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 0a28124..cfdfae5 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -118,6 +118,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
+ int sequence_data; /* dump sequence data even in schema-only mode */
} RestoreOptions;
typedef struct _dumpOptions
@@ -160,6 +161,8 @@ typedef struct _dumpOptions
bool outputBlobs;
int outputNoOwner;
char *outputSuperuser;
+
+ int sequence_data; /* dump sequence data even in schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index a69b06f..aeab545 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -166,6 +166,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->lockWaitTimeout = ropt->lockWaitTimeout;
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
+ dopt->sequence_data = ropt->sequence_data;
return dopt;
}
@@ -2831,7 +2832,10 @@ _tocEntryRequired(TocEntry *te, teSection curSection, RestoreOptions *ropt)
/* Mask it if we only want schema */
if (ropt->schemaOnly)
- res = res & REQ_SCHEMA;
+ {
+ if (!(ropt->sequence_data && strcmp(te->desc, "SEQUENCE SET") == 0))
+ res = res & REQ_SCHEMA;
+ }
/* Mask it if we only want data */
if (ropt->dataOnly)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 004499c..413392b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -219,7 +219,7 @@ static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
-static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids);
+static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo, bool oids);
static void buildMatViewRefreshDependencies(Archive *fout);
static void getTableDataFKConstraints(void);
@@ -556,6 +556,12 @@ main(int argc, char **argv)
if (dopt.column_inserts)
dopt.dump_inserts = 1;
+ /* Binary upgrade mode implies dumping sequence data even in schema-only
+ * mode. This is not exposed as a separate option, but kept separate
+ * internally for clarity. */
+ if (dopt.binary_upgrade)
+ dopt.sequence_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -746,12 +752,15 @@ main(int argc, char **argv)
if (!dopt.schemaOnly)
{
- getTableData(&dopt, tblinfo, numTables, dopt.oids);
+ getTableData(&dopt, tblinfo, numTables, dopt.oids, 0);
buildMatViewRefreshDependencies(fout);
if (dopt.dataOnly)
getTableDataFKConstraints();
}
+ if (dopt.schemaOnly && dopt.sequence_data)
+ getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+
if (dopt.outputBlobs)
getBlobs(fout);
@@ -835,6 +844,7 @@ main(int argc, char **argv)
ropt->lockWaitTimeout = dopt.lockWaitTimeout;
ropt->include_everything = dopt.include_everything;
ropt->enable_row_security = dopt.enable_row_security;
+ ropt->sequence_data = dopt.sequence_data;
if (compressLevel == -1)
ropt->compression = 0;
@@ -2090,13 +2100,14 @@ refreshMatViewData(Archive *fout, TableDataInfo *tdinfo)
* set up dumpable objects representing the contents of tables
*/
static void
-getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids)
+getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids, char relkind)
{
int i;
for (i = 0; i < numTables; i++)
{
- if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA)
+ if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA &&
+ (!relkind || tblinfo[i].relkind == relkind))
makeTableDataInfo(dopt, &(tblinfo[i]), oids);
}
}
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 1200c7f..6ea5720 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -444,7 +444,7 @@ get_rel_infos(ClusterInfo *cluster, DbInfo *dbinfo)
" SELECT c.oid, 0::oid, 0::oid "
" FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n "
" ON c.relnamespace = n.oid "
- " WHERE relkind IN ('r', 'm', 'S') AND "
+ " WHERE relkind IN ('r', 'm') AND "
/* exclude possible orphaned temp tables */
" ((n.nspname !~ '^pg_temp_' AND "
" n.nspname !~ '^pg_toast_temp_' AND "
--
2.10.0
23.09.2016 21:06, Peter Eisentraut:
Here is an updated patch set. Compared to the initial set, I have
changed pg_dump's sorting priorities so that sequence data is always
after table data. This would otherwise have introduced a problem
because sortDataAndIndexObjectsBySize() only considers consecutive
DO_TABLE_DATA entries. Also, I have removed the separate
--sequence-data switch from pg_dump and made it implicit in
--binary-upgrade. (So the previous patches 0002 and 0003 have been
combined, because it's no longer a separate feature.)
The patches are good, no complaints.
But again, I have the same question.
I was confused, why do we always dump sequence data,
because I'd overlooked the --sequence-data key. I'd rather leave this
option,
because it's quite non intuitive behaviour...
/* dump sequence data even in schema-only mode */
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Oct 1, 2016 at 1:50 AM, Anastasia Lubennikova
<a.lubennikova@postgrespro.ru> wrote:
23.09.2016 21:06, Peter Eisentraut:
Here is an updated patch set. Compared to the initial set, I have
changed pg_dump's sorting priorities so that sequence data is always
after table data. This would otherwise have introduced a problem
because sortDataAndIndexObjectsBySize() only considers consecutive
DO_TABLE_DATA entries. Also, I have removed the separate
--sequence-data switch from pg_dump and made it implicit in
--binary-upgrade. (So the previous patches 0002 and 0003 have been
combined, because it's no longer a separate feature.)The patches are good, no complaints.
But again, I have the same question.
I was confused, why do we always dump sequence data,
because I'd overlooked the --sequence-data key. I'd rather leave this
option,
because it's quite non intuitive behaviour...
/* dump sequence data even in schema-only mode */
Moved to next CF. This is fresh.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 9/30/16 12:50 PM, Anastasia Lubennikova wrote:
The patches are good, no complaints.
But again, I have the same question.
I was confused, why do we always dump sequence data,
because I'd overlooked the --sequence-data key. I'd rather leave this
option,
because it's quite non intuitive behaviour...
/* dump sequence data even in schema-only mode */
Here are rebased patches.
Regarding your question: The initial patch had a separate option for
this behavior, which was then used by pg_upgrade. It was commented that
this option is not useful outside of pg_upgrade, so it doesn't need to
be exposed as a user-facing option. I agreed with that and removed the
option. We can always add the option back easily if someone really
wants it, but so far no use case has been presented. So I suggest we
proceed with this proposal ignoring whether this option is exposed or not.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
v3-0001-pg_dump-Separate-table-and-sequence-data-object-t.patchtext/x-patch; name=v3-0001-pg_dump-Separate-table-and-sequence-data-object-t.patchDownload
From d2b98ba5df815018dac1650134398c1bac7164a4 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH v3 1/2] pg_dump: Separate table and sequence data object types
Instead of handling both sequence data and table data internally as
"table data", handle sequences separately under a "sequence set" type.
We already handled materialized view data differently, so it makes the
code somewhat cleaner to handle each relation kind separately at the top
level.
This does not change the output format, since there already was a
separate "SEQUENCE SET" archive entry type. A noticeable difference is
that SEQUENCE SET entries now always appear after TABLE DATA entries.
And in parallel mode there is less sorting to do, because the sequence
data entries are no longer considered table data.
---
src/bin/pg_dump/pg_dump.c | 11 +++++++----
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/pg_dump_sort.c | 28 +++++++++++++++++-----------
3 files changed, 25 insertions(+), 15 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4da297f..3485cab 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -2090,6 +2090,8 @@ makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo, bool oids)
if (tbinfo->relkind == RELKIND_MATVIEW)
tdinfo->dobj.objType = DO_REFRESH_MATVIEW;
+ else if (tbinfo->relkind == RELKIND_SEQUENCE)
+ tdinfo->dobj.objType = DO_SEQUENCE_SET;
else
tdinfo->dobj.objType = DO_TABLE_DATA;
@@ -8498,11 +8500,11 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_TRANSFORM:
dumpTransform(fout, (TransformInfo *) dobj);
break;
+ case DO_SEQUENCE_SET:
+ dumpSequenceData(fout, (TableDataInfo *) dobj);
+ break;
case DO_TABLE_DATA:
- if (((TableDataInfo *) dobj)->tdtable->relkind == RELKIND_SEQUENCE)
- dumpSequenceData(fout, (TableDataInfo *) dobj);
- else
- dumpTableData(fout, (TableDataInfo *) dobj);
+ dumpTableData(fout, (TableDataInfo *) dobj);
break;
case DO_DUMMY_TYPE:
/* table rowtypes and array types are never dumped separately */
@@ -16226,6 +16228,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
addObjectDependency(preDataBound, dobj->dumpId);
break;
case DO_TABLE_DATA:
+ case DO_SEQUENCE_SET:
case DO_BLOB_DATA:
/* Data objects: must come between the boundaries */
addObjectDependency(dobj, preDataBound->dumpId);
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a60cf95..642c4d5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -63,6 +63,7 @@ typedef enum
DO_PROCLANG,
DO_CAST,
DO_TABLE_DATA,
+ DO_SEQUENCE_SET,
DO_DUMMY_TYPE,
DO_TSPARSER,
DO_TSDICT,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 195b84a..5b96334 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -47,14 +47,15 @@ static const int dbObjectTypePriority[] =
11, /* DO_CONVERSION */
18, /* DO_TABLE */
20, /* DO_ATTRDEF */
- 27, /* DO_INDEX */
- 28, /* DO_RULE */
- 29, /* DO_TRIGGER */
- 26, /* DO_CONSTRAINT */
- 30, /* DO_FK_CONSTRAINT */
+ 28, /* DO_INDEX */
+ 29, /* DO_RULE */
+ 30, /* DO_TRIGGER */
+ 27, /* DO_CONSTRAINT */
+ 31, /* DO_FK_CONSTRAINT */
2, /* DO_PROCLANG */
10, /* DO_CAST */
23, /* DO_TABLE_DATA */
+ 24, /* DO_SEQUENCE_SET */
19, /* DO_DUMMY_TYPE */
12, /* DO_TSPARSER */
14, /* DO_TSDICT */
@@ -62,15 +63,15 @@ static const int dbObjectTypePriority[] =
15, /* DO_TSCONFIG */
16, /* DO_FDW */
17, /* DO_FOREIGN_SERVER */
- 31, /* DO_DEFAULT_ACL */
+ 32, /* DO_DEFAULT_ACL */
3, /* DO_TRANSFORM */
21, /* DO_BLOB */
- 24, /* DO_BLOB_DATA */
+ 25, /* DO_BLOB_DATA */
22, /* DO_PRE_DATA_BOUNDARY */
- 25, /* DO_POST_DATA_BOUNDARY */
- 32, /* DO_EVENT_TRIGGER */
- 33, /* DO_REFRESH_MATVIEW */
- 34 /* DO_POLICY */
+ 26, /* DO_POST_DATA_BOUNDARY */
+ 33, /* DO_EVENT_TRIGGER */
+ 34, /* DO_REFRESH_MATVIEW */
+ 35 /* DO_POLICY */
};
static DumpId preDataBoundId;
@@ -1345,6 +1346,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"TABLE DATA %s (ID %d OID %u)",
obj->name, obj->dumpId, obj->catId.oid);
return;
+ case DO_SEQUENCE_SET:
+ snprintf(buf, bufsize,
+ "SEQUENCE SET %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
case DO_DUMMY_TYPE:
snprintf(buf, bufsize,
"DUMMY TYPE %s (ID %d OID %u)",
--
2.10.2
v3-0002-pg_upgrade-Upgrade-sequence-data-via-pg_dump.patchtext/x-patch; name=v3-0002-pg_upgrade-Upgrade-sequence-data-via-pg_dump.patchDownload
From bf17e1aface2dbdcbdb72cbdaa45ca5bf8e725f0 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 23 Aug 2016 12:00:00 -0400
Subject: [PATCH v3 2/2] pg_upgrade: Upgrade sequence data via pg_dump
Previously, pg_upgrade migrated sequence data like tables by copying the
on-disk file. This does not allow any changes in the on-disk format for
sequences. It's simpler to just have pg_dump set the new sequence
values as it normally does. To do that, create a hidden submode in
pg_dump that dumps sequence data even when a schema-only dump is
requested, and trigger that submode in binary upgrade mode. (This new
submode could easily be exposed as a command-line option, but it has
limited use outside of pg_dump and would probably cause some confusion,
so we don't do that at this time.)
---
src/bin/pg_dump/pg_backup.h | 3 +++
src/bin/pg_dump/pg_backup_archiver.c | 6 +++++-
src/bin/pg_dump/pg_dump.c | 19 +++++++++++++++----
src/bin/pg_upgrade/info.c | 2 +-
4 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 0a28124..cfdfae5 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -118,6 +118,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
+ int sequence_data; /* dump sequence data even in schema-only mode */
} RestoreOptions;
typedef struct _dumpOptions
@@ -160,6 +161,8 @@ typedef struct _dumpOptions
bool outputBlobs;
int outputNoOwner;
char *outputSuperuser;
+
+ int sequence_data; /* dump sequence data even in schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 0e20985..b938d79 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -171,6 +171,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->lockWaitTimeout = ropt->lockWaitTimeout;
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
+ dopt->sequence_data = ropt->sequence_data;
return dopt;
}
@@ -2855,7 +2856,10 @@ _tocEntryRequired(TocEntry *te, teSection curSection, RestoreOptions *ropt)
/* Mask it if we only want schema */
if (ropt->schemaOnly)
- res = res & REQ_SCHEMA;
+ {
+ if (!(ropt->sequence_data && strcmp(te->desc, "SEQUENCE SET") == 0))
+ res = res & REQ_SCHEMA;
+ }
/* Mask it if we only want data */
if (ropt->dataOnly)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3485cab..ee1f673 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -216,7 +216,7 @@ static void addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
DumpableObject *boundaryObjs);
static void getDomainConstraints(Archive *fout, TypeInfo *tyinfo);
-static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids);
+static void getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids, char relkind);
static void makeTableDataInfo(DumpOptions *dopt, TableInfo *tbinfo, bool oids);
static void buildMatViewRefreshDependencies(Archive *fout);
static void getTableDataFKConstraints(void);
@@ -546,6 +546,12 @@ main(int argc, char **argv)
if (dopt.column_inserts)
dopt.dump_inserts = 1;
+ /* Binary upgrade mode implies dumping sequence data even in schema-only
+ * mode. This is not exposed as a separate option, but kept separate
+ * internally for clarity. */
+ if (dopt.binary_upgrade)
+ dopt.sequence_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -722,12 +728,15 @@ main(int argc, char **argv)
if (!dopt.schemaOnly)
{
- getTableData(&dopt, tblinfo, numTables, dopt.oids);
+ getTableData(&dopt, tblinfo, numTables, dopt.oids, 0);
buildMatViewRefreshDependencies(fout);
if (dopt.dataOnly)
getTableDataFKConstraints();
}
+ if (dopt.schemaOnly && dopt.sequence_data)
+ getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+
if (dopt.outputBlobs)
getBlobs(fout);
@@ -806,6 +815,7 @@ main(int argc, char **argv)
ropt->lockWaitTimeout = dopt.lockWaitTimeout;
ropt->include_everything = dopt.include_everything;
ropt->enable_row_security = dopt.enable_row_security;
+ ropt->sequence_data = dopt.sequence_data;
if (compressLevel == -1)
ropt->compression = 0;
@@ -2039,13 +2049,14 @@ refreshMatViewData(Archive *fout, TableDataInfo *tdinfo)
* set up dumpable objects representing the contents of tables
*/
static void
-getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids)
+getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids, char relkind)
{
int i;
for (i = 0; i < numTables; i++)
{
- if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA)
+ if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA &&
+ (!relkind || tblinfo[i].relkind == relkind))
makeTableDataInfo(dopt, &(tblinfo[i]), oids);
}
}
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 1200c7f..6ea5720 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -444,7 +444,7 @@ get_rel_infos(ClusterInfo *cluster, DbInfo *dbinfo)
" SELECT c.oid, 0::oid, 0::oid "
" FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n "
" ON c.relnamespace = n.oid "
- " WHERE relkind IN ('r', 'm', 'S') AND "
+ " WHERE relkind IN ('r', 'm') AND "
/* exclude possible orphaned temp tables */
" ((n.nspname !~ '^pg_temp_' AND "
" n.nspname !~ '^pg_toast_temp_' AND "
--
2.10.2
On Mon, Oct 31, 2016 at 9:53 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 9/30/16 12:50 PM, Anastasia Lubennikova wrote:
The patches are good, no complaints.
But again, I have the same question.
I was confused, why do we always dump sequence data,
because I'd overlooked the --sequence-data key. I'd rather leave this
option,
because it's quite non intuitive behaviour...
/* dump sequence data even in schema-only mode */Here are rebased patches.
Regarding your question: The initial patch had a separate option for
this behavior, which was then used by pg_upgrade. It was commented that
this option is not useful outside of pg_upgrade, so it doesn't need to
be exposed as a user-facing option. I agreed with that and removed the
option. We can always add the option back easily if someone really
wants it, but so far no use case has been presented. So I suggest we
proceed with this proposal ignoring whether this option is exposed or not.
I had a look at those fresh patches, and 0001 looks like a good thing.
This makes the separation between sequences and table data dump
cleaner. I ran some tests with pg_upgrade and 0002, and things are
clear. And +1 for the way done in the patch, aka no options of pg_dump
exposed to user, still keep the option tracking as a separate value.
One small thing here:
static void
-getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables, bool oids)
+getTableData(DumpOptions *dopt, TableInfo *tblinfo, int numTables,
bool oids, char relkind)
{
int i;
for (i = 0; i < numTables; i++)
{
- if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA)
+ if (tblinfo[i].dobj.dump & DUMP_COMPONENT_DATA &&
+ (!relkind || tblinfo[i].relkind == relkind))
makeTableDataInfo(dopt, &(tblinfo[i]), oids)
One idea here would be to have an extra routine, getSequenceData and
not extend getTableData() with relkind as extra argument. I am fine
with the way patch does things, so I just switched the patch as ready
for committer.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 11/2/16 2:34 AM, Michael Paquier wrote:
I had a look at those fresh patches, and 0001 looks like a good thing.
This makes the separation between sequences and table data dump
cleaner. I ran some tests with pg_upgrade and 0002, and things are
clear. And +1 for the way done in the patch, aka no options of pg_dump
exposed to user, still keep the option tracking as a separate value.
Committed, thanks.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers