partial data migration
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?
Thank you!
On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to
another. What would be the easiest way to do it? In Oracle, I would set up
dblink. What about postgresql?
postgres_fdw
--
Angular momentum makes the world go 'round.
Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option?
Sent from my iPhone
On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com<mailto:ronljohnsonjr@gmail.com>> wrote:
On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?
postgres_fdw
--
Angular momentum makes the world go 'round.
(8.3? That's even older than what we just migrated from!!!)
No. Make some views (I'd probably make them "month-sized"), COPY each view
from the source db to a file, and then COPY each file to it's relevant
target table.
You should also think about a program named pg_bulkload.
https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.html
On 3/7/19 7:53 PM, Julie Nishimura wrote:
Thank you Ron! What if dev environment is on 9.6, but prod is on version
8.3? Will posgtres_fdw still be the right option?Sent from my iPhone
On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com
<mailto:ronljohnsonjr@gmail.com>> wrote:On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to
another. What would be the easiest way to do it? In Oracle, I would set
up dblink. What about postgresql?postgres_fdw
--
Angular momentum makes the world go 'round.
Ron, thanksagain. In case if I need to migrate the entire tables, I should be able to use pg_dump and pg_restore for certain tables, even between different versions, right? In case if I need to migrate from 8 to 9?
Thanks
________________________________
From: Ron <ronljohnsonjr@gmail.com>
Sent: Thursday, March 7, 2019 5:59 PM
To: pgsql-general@lists.postgresql.org
Subject: Re: partial data migration
(8.3? That's even older than what we just migrated from!!!)
No. Make some views (I'd probably make them "month-sized"), COPY each view from the source db to a file, and then COPY each file to it's relevant target table.
You should also think about a program named pg_bulkload.
https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.html
On 3/7/19 7:53 PM, Julie Nishimura wrote:
Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option?
Sent from my iPhone
On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com<mailto:ronljohnsonjr@gmail.com>> wrote:
On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?
postgres_fdw
--
Angular momentum makes the world go 'round.
Yes, for whole tables (even sets of tables) "pg_dump --table=" is good at
that. Even better, you can run the 9.6 pg_dump against the 8.3 database and
get parallelism with "--jobs".
On 3/7/19 8:11 PM, Julie Nishimura wrote:
Ron, thanksagain. In case if I need to migrate the entire tables, I should
be able to use pg_dump and pg_restore for certain tables, even between
different versions, right? In case if I need to migrate from 8 to 9?Thanks
----------------------------------------------------------------------------
*From:* Ron <ronljohnsonjr@gmail.com>
*Sent:* Thursday, March 7, 2019 5:59 PM
*To:* pgsql-general@lists.postgresql.org
*Subject:* Re: partial data migration
(8.3? That's even older than what we just migrated from!!!)No. Make some views (I'd probably make them "month-sized"), COPY each
view from the source db to a file, and then COPY each file to it's
relevant target table.You should also think about a program named pg_bulkload.
https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.htmlOn 3/7/19 7:53 PM, Julie Nishimura wrote:
Thank you Ron! What if dev environment is on 9.6, but prod is on version
8.3? Will posgtres_fdw still be the right option?Sent from my iPhone
On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com
<mailto:ronljohnsonjr@gmail.com>> wrote:On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to
another. What would be the easiest way to do it? In Oracle, I would set
up dblink. What about postgresql?postgres_fdw
--
Angular momentum makes the world go 'round.
--
Angular momentum makes the world go 'round.
Great, thanks!
Sent from my iPhone
On Mar 7, 2019, at 7:48 PM, Ron <ronljohnsonjr@gmail.com<mailto:ronljohnsonjr@gmail.com>> wrote:
Yes, for whole tables (even sets of tables) "pg_dump --table=" is good at that. Even better, you can run the 9.6 pg_dump against the 8.3 database and get parallelism with "--jobs".
On 3/7/19 8:11 PM, Julie Nishimura wrote:
Ron, thanksagain. In case if I need to migrate the entire tables, I should be able to use pg_dump and pg_restore for certain tables, even between different versions, right? In case if I need to migrate from 8 to 9?
Thanks
________________________________
From: Ron <ronljohnsonjr@gmail.com><mailto:ronljohnsonjr@gmail.com>
Sent: Thursday, March 7, 2019 5:59 PM
To: pgsql-general@lists.postgresql.org<mailto:pgsql-general@lists.postgresql.org>
Subject: Re: partial data migration
(8.3? That's even older than what we just migrated from!!!)
No. Make some views (I'd probably make them "month-sized"), COPY each view from the source db to a file, and then COPY each file to it's relevant target table.
You should also think about a program named pg_bulkload.
https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.html
On 3/7/19 7:53 PM, Julie Nishimura wrote:
Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option?
Sent from my iPhone
On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com<mailto:ronljohnsonjr@gmail.com>> wrote:
On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?
postgres_fdw
--
Angular momentum makes the world go 'round.
--
Angular momentum makes the world go 'round.
On 3/7/19 5:53 PM, Julie Nishimura wrote:
Thank you Ron! What if dev environment is on 9.6, but prod is on version
8.3? Will posgtres_fdw still be the right option?
Thanks to the documentation writers:
https://www.postgresql.org/docs/11/postgres-fdw.html#id-1.11.7.42.15
Sent from my iPhone
On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com
<mailto:ronljohnsonjr@gmail.com>> wrote:On 3/7/19 1:54 AM, Julie Nishimura wrote:
Hello psql friends,
We need to migrate only 6 months worth of data from one instance to
another. What would be the easiest way to do it? In Oracle, I would
set up dblink. What about postgresql?postgres_fdw
--
Angular momentum makes the world go 'round.
--
Adrian Klaver
adrian.klaver@aklaver.com