Ora2pg Delta Migration: Oracle to PostgreSQL

Started by Amit Sharmaalmost 2 years ago5 messagesgeneral
Jump to latest
#1Amit Sharma
amitpgsql@gmail.com

Hello,

Has anyone tried delta/incremental data migration for Oracle to PostgreSQL
using Ora2pg? Or what are the best options to run delta migration for
Oracle to PostgreSQL?

Thanks
Amit

#2Ron
ronljohnsonjr@gmail.com
In reply to: Amit Sharma (#1)
Re: Ora2pg Delta Migration: Oracle to PostgreSQL

On Thu, May 2, 2024 at 8:28 PM Amit Sharma <amitpgsql@gmail.com> wrote:

Hello,

Has anyone tried delta/incremental data migration for Oracle to PostgreSQL
using Ora2pg? Or what are the best options to run delta migration for
Oracle to PostgreSQL?

What do the ora2pg docs say about whether or not that feature is
implemented? (It wasn't when I last used it in 2022.)

#3Muhammad Ikram
mmikram@gmail.com
In reply to: Ron (#2)
Re: Ora2pg Delta Migration: Oracle to PostgreSQL

Hi,

Not related to Ora2PG but in the past I have used the EDB Migration toolkit
for such scenarios. Filterprops option can help in specifying filter (where
clause).

Regards,
Ikram

On Fri, May 3, 2024 at 8:12 AM Ron Johnson <ronljohnsonjr@gmail.com> wrote:

On Thu, May 2, 2024 at 8:28 PM Amit Sharma <amitpgsql@gmail.com> wrote:

Hello,

Has anyone tried delta/incremental data migration for Oracle to
PostgreSQL using Ora2pg? Or what are the best options to run delta
migration for Oracle to PostgreSQL?

What do the ora2pg docs say about whether or not that feature is
implemented? (It wasn't when I last used it in 2022.)

--
Muhammad Ikram

#4Avinash Kumar
avinash.vallarapu@gmail.com
In reply to: Muhammad Ikram (#3)
Re: Ora2pg Delta Migration: Oracle to PostgreSQL

Hi,

Has anyone tried delta/incremental data migration for Oracle to PostgreSQL

using Ora2pg? Or what are the best options to run delta migration for
Oracle to PostgreSQL?

There are a few ways of dealing with it when using Ora2Pg.

*Option 1 : *Ora2Pg configuration file allows us to pass the
conditions(predicates) to be applied to a table while migrating the data
from a table. So, if there is a way to filter the data since the last full
load, you can pass the condition to the configuration file, for each table.
This is some work for sure but it works great if you are able to identify
those conditions such as creation_date or last_updated_date, let's say.

*Option2 :* Ora2Pg is currently CDC ready. What this means is that, Ora2Pg
can provide the SCN at which the table copy has been initiated.
Technically, we provide the SCN to the CDC tools or custom CDC techniques
to continue replication since that SCN.

*Option 2.1 :* Leverage Debezium, an Open Source solution to perform
continuous replication from Oracle to PostgreSQL from the SCN produced by
Ora2Pg for that table.

So, it is all about Ora2Pg + Debezium

It might be initially challenging, but you will get there.

Regards,
Avi Vallarapu,
CEO,
HexaCluster Corp.

On Fri, May 3, 2024 at 3:25 AM Muhammad Ikram <mmikram@gmail.com> wrote:

Show quoted text

Hi,

Not related to Ora2PG but in the past I have used the EDB Migration
toolkit for such scenarios. Filterprops option can help in specifying
filter (where clause).

Regards,
Ikram

On Fri, May 3, 2024 at 8:12 AM Ron Johnson <ronljohnsonjr@gmail.com>
wrote:

On Thu, May 2, 2024 at 8:28 PM Amit Sharma <amitpgsql@gmail.com> wrote:

Hello,

Has anyone tried delta/incremental data migration for Oracle to
PostgreSQL using Ora2pg? Or what are the best options to run delta
migration for Oracle to PostgreSQL?

What do the ora2pg docs say about whether or not that feature is
implemented? (It wasn't when I last used it in 2022.)

--
Muhammad Ikram

#5Amit Sharma
amitpgsql@gmail.com
In reply to: Avinash Kumar (#4)
Re: Ora2pg Delta Migration: Oracle to PostgreSQL

Thanks Avinash. Appriciate the help!!

Regards
Amit

On Fri, May 3, 2024 at 5:38 AM Avinash Vallarapu <
avinash.vallarapu@gmail.com> wrote:

Show quoted text

Hi,

Has anyone tried delta/incremental data migration for Oracle to PostgreSQL

using Ora2pg? Or what are the best options to run delta migration for
Oracle to PostgreSQL?

There are a few ways of dealing with it when using Ora2Pg.

*Option 1 : *Ora2Pg configuration file allows us to pass the
conditions(predicates) to be applied to a table while migrating the data
from a table. So, if there is a way to filter the data since the last full
load, you can pass the condition to the configuration file, for each table.
This is some work for sure but it works great if you are able to identify
those conditions such as creation_date or last_updated_date, let's say.

*Option2 :* Ora2Pg is currently CDC ready. What this means is that,
Ora2Pg can provide the SCN at which the table copy has been initiated.
Technically, we provide the SCN to the CDC tools or custom CDC techniques
to continue replication since that SCN.

*Option 2.1 :* Leverage Debezium, an Open Source solution to perform
continuous replication from Oracle to PostgreSQL from the SCN produced by
Ora2Pg for that table.

So, it is all about Ora2Pg + Debezium

It might be initially challenging, but you will get there.

Regards,
Avi Vallarapu,
CEO,
HexaCluster Corp.

On Fri, May 3, 2024 at 3:25 AM Muhammad Ikram <mmikram@gmail.com> wrote:

Hi,

Not related to Ora2PG but in the past I have used the EDB Migration
toolkit for such scenarios. Filterprops option can help in specifying
filter (where clause).

Regards,
Ikram

On Fri, May 3, 2024 at 8:12 AM Ron Johnson <ronljohnsonjr@gmail.com>
wrote:

On Thu, May 2, 2024 at 8:28 PM Amit Sharma <amitpgsql@gmail.com> wrote:

Hello,

Has anyone tried delta/incremental data migration for Oracle to
PostgreSQL using Ora2pg? Or what are the best options to run delta
migration for Oracle to PostgreSQL?

What do the ora2pg docs say about whether or not that feature is
implemented? (It wasn't when I last used it in 2022.)

--
Muhammad Ikram