Restoring tables with circular references dumped to separate files

Started by doganmehover 8 years ago7 messagesgeneral
Jump to latest
#1doganmeh
mehmet@edgle.com

I was dumping each table to a separate file so I could pick and choose when
restoring. However, seems this was not a great idea, since two of my tables
happened to reference each other via FOREIGN KEYs, and I am not able to
restore them. Is there a way to do this without manually merging the dump
files? Thanks for guidance in advance.

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2Uwe C. Schroeder
uwe@oss4u.com
In reply to: doganmeh (#1)
Re: Restoring tables with circular references dumped to separate files

I would alter the tables and remove the foreign key constraint, restore the
data and add the constraint back. If the data is consistent, adding the
foreign key should work without error.

Show quoted text

On Friday, October 20, 2017 8:15:27 PM PDT doganmeh wrote:

I was dumping each table to a separate file so I could pick and choose when
restoring. However, seems this was not a great idea, since two of my tables
happened to reference each other via FOREIGN KEYs, and I am not able to
restore them. Is there a way to do this without manually merging the dump
files? Thanks for guidance in advance.

--
Sent from:
http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

#3doganmeh
mehmet@edgle.com
In reply to: Uwe C. Schroeder (#2)
Re: Restoring tables with circular references dumped to separate files

Seems that would be easier and less error prone. Thanks,

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Melvin Davidson
melvin6925@gmail.com
In reply to: doganmeh (#3)
Re: Re: Restoring tables with circular references dumped to separate files

On Sat, Oct 21, 2017 at 8:24 AM, doganmeh <mehmet@edgle.com> wrote:

Seems that would be easier and less error prone. Thanks,

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-
f1843780.html

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

I was dumping each table to a separate file so I could pick and choose

when restoring...

*It seems to me that instead on multiple single table dumps, you could take
advantage of restoring from a list-file.*

*Just *
*1. do a dump with custom format ( -F c)*

*EG: pg_dump -U postgres -F c -t table1 -t table2 -t table3 yourdb >
yourdb.dmp*

*2. use pg_restore -l to create a list-file*
*EG: pg_restore -l yourdb.dmp > yourdb.lis*

*3. edit yourdb.lis and comment out ( prefix with ; ) any line you don't
want*

*4. then use pg_restore with the edited list-file *

*EG: pg_restore -L yourdb.lis yourdb.dmp*

*See examples at the end of*

*https://www.postgresql.org/docs/9.6/static/app-pgrestore.html
<https://www.postgresql.org/docs/9.6/static/app-pgrestore.html&gt;*

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

#5doganmeh
mehmet@edgle.com
In reply to: Melvin Davidson (#4)
Re: Restoring tables with circular references dumped to separate files

The list approach for partial restore is also useful, thank you.

On another note, I used to take full backups (entire database), however
switched to table by table scheme in order to make it more VCS friendly.
Namely, so I only check into github the dumps of the tables that are updated
only.

So, from that perspective, is there a dump-restore scenario that is widely
used, but is also VCS friendly? To my knowledge, pg_restore does not restore
backups that are in "plain text" format, and compressed formats such as
"tar" would not be github friendly.

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Melvin Davidson
melvin6925@gmail.com
In reply to: doganmeh (#5)
Re: Re: Restoring tables with circular references dumped to separate files

On Sat, Oct 21, 2017 at 4:48 PM, doganmeh <mehmet@edgle.com> wrote:

The list approach for partial restore is also useful, thank you.

On another note, I used to take full backups (entire database), however
switched to table by table scheme in order to make it more VCS friendly.
Namely, so I only check into github the dumps of the tables that are
updated
only.

So, from that perspective, is there a dump-restore scenario that is widely
used, but is also VCS friendly? To my knowledge, pg_restore does not
restore
backups that are in "plain text" format, and compressed formats such as
"tar" would not be github friendly.

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-
f1843780.html

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

*>...is there a dump-restore scenario that is widely used, but is also VCS
friendly?*

*You might want to give consideration to pg_extractor.*

*https://github.com/omniti-labs/pg_extractor
<https://github.com/omniti-labs/pg_extractor&gt;*

*https://www.keithf4.com/pg_extractor/
<https://www.keithf4.com/pg_extractor/&gt;*

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

#7Francisco Olarte
folarte@peoplecall.com
In reply to: doganmeh (#5)
Re: Re: Restoring tables with circular references dumped to separate files

On Sat, Oct 21, 2017 at 10:48 PM, doganmeh <mehmet@edgle.com> wrote:
...

On another note, I used to take full backups (entire database), however
switched to table by table scheme in order to make it more VCS friendly.
Namely, so I only check into github the dumps of the tables that are updated
only.
So, from that perspective, is there a dump-restore scenario that is widely
used, but is also VCS friendly? To my knowledge, pg_restore does not restore
backups that are in "plain text" format, and compressed formats such as
"tar" would not be github friendly.

Not widely used, but you have the directory format ( disclaimer: have
not tested it for VCS friendliness ). It populates a directory similar
to what uncompressing a tar format would, but I do not know if it
renames the files from run to run, but should be easy to test.

Also note it is documented as compressed BY DEFAULT, but you can use
options to avoid compression, and it is the only one which supports
paralell dumps.

Also, custom and tar can be made uncompressed, but I do not think
that's a great idea.

Francisco Olarte.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general