How long does iteration over 4-5 million rows usually take?

Started by Shaozhong SHIabout 4 years ago9 messagesgeneral
Jump to latest
#1Shaozhong SHI
shishaozhong@gmail.com

I have a script running to iterate over 4-5 million rows. It keeps showing
up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

Regards,

David

#2Shaozhong SHI
shishaozhong@gmail.com
In reply to: Shaozhong SHI (#1)

I have a script running to iterate over 4-5 million rows. It keeps showing
up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

Regards,

David

#3Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Shaozhong SHI (#1)
Re: How long does iteration over 4-5 million rows usually take?

On 4/1/22 18:34, Shaozhong SHI wrote:

I have a script running to iterate over 4-5 million rows.  It keeps
showing up in red in PgAdmin.  It remains active.

How long does iteration over 4-5 million rows usually take?

Given that there is no real information provided in the problem
description there is no chance for an answer more detailed then; as long
as it takes.

Regards,

David

--
Adrian Klaver
adrian.klaver@aklaver.com

#4Ron
ronljohnsonjr@gmail.com
In reply to: Shaozhong SHI (#1)
Re: How long does iteration over 4-5 million rows usually take?

On 4/1/22 20:34, Shaozhong SHI wrote:

I have a script running to iterate over 4-5 million rows.  It keeps
showing up in red in PgAdmin.  It remains active.

How long does iteration over 4-5 million rows usually take?

What /*exactly*/ are you doing?

--
Angular momentum makes the world go 'round.

#5Rob Sargent
robjsargent@gmail.com
In reply to: Ron (#4)
Re: How long does iteration over 4-5 million rows usually take?

On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr@gmail.com> wrote:

 On 4/1/22 20:34, Shaozhong SHI wrote:

I have a script running to iterate over 4-5 million rows. It keeps showing up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

4-5 million times as long as it takes to do one iteration ( if you’re doing it correctly)

#6Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Rob Sargent (#5)
Aw: Re: How long does iteration over 4-5 million rows usually take?

On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr@gmail.com> wrote:

 On 4/1/22 20:34, Shaozhong SHI wrote:

I have a script running to iterate over 4-5 million rows. It keeps showing up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

4-5 million times as long as it takes to do one iteration ( if you’re doing it correctly)

I may not take quite that long because setup/teardown times might not be needed for each iteration.

Best,
Karsten

#7Shaozhong SHI
shishaozhong@gmail.com
In reply to: Karsten Hilbert (#6)
Re: Re: How long does iteration over 4-5 million rows usually take?

Thanks, Karsten,

I would like the information to work planning purpose.

Regards,

David

On Sat, 2 Apr 2022 at 14:47, Karsten Hilbert <Karsten.Hilbert@gmx.net>
wrote:

Show quoted text

On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr@gmail.com> wrote:

 On 4/1/22 20:34, Shaozhong SHI wrote:

I have a script running to iterate over 4-5 million rows. It keeps

showing up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

4-5 million times as long as it takes to do one iteration ( if you’re

doing it correctly)

I may not take quite that long because setup/teardown times might not be
needed for each iteration.

Best,
Karsten

#8Benedict Holland
benedict.m.holland@gmail.com
In reply to: Shaozhong SHI (#7)
Re: Re: How long does iteration over 4-5 million rows usually take?

For work planning, create a temporary table, copy from a generated data
set, and test it out. It really depends. Normally, you don't iterate in SQL
since rows are assumed to be independent. You think about SQL in terms of
sets and let application code handle the data transfers. It also really
depends on the size of your tables, indexing, joins, and more. Basically,
there isn't really an answer but it isnt hard to test.

Thanks,
Ben

On Sat, Apr 2, 2022, 10:16 AM Shaozhong SHI <shishaozhong@gmail.com> wrote:

Show quoted text

Thanks, Karsten,

I would like the information to work planning purpose.

Regards,

David

On Sat, 2 Apr 2022 at 14:47, Karsten Hilbert <Karsten.Hilbert@gmx.net>
wrote:

On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr@gmail.com> wrote:

 On 4/1/22 20:34, Shaozhong SHI wrote:

I have a script running to iterate over 4-5 million rows. It keeps

showing up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

4-5 million times as long as it takes to do one iteration ( if you’re

doing it correctly)

I may not take quite that long because setup/teardown times might not be
needed for each iteration.

Best,
Karsten

#9Steve Midgley
science@misuse.org
In reply to: Shaozhong SHI (#2)
Re: How long does iteration over 4-5 million rows usually take?

On Fri, Apr 1, 2022 at 6:35 PM Shaozhong SHI <shishaozhong@gmail.com> wrote:

I have a script running to iterate over 4-5 million rows. It keeps
showing up in red in PgAdmin. It remains active.

How long does iteration over 4-5 million rows usually take?

Please share the SQL query, schema, and sample data. If you are using a
third party script with a db connector, then the answer is much harder to
answer.