pg_migrator alpha 5 - truncates at 10 M rows
2009.05.09
pg_migrator alpha 5 results
from PostgreSQL 8.3.7 to 8.4cvs
Centos 5.2 x86_64 GNU/Linux
pg_migrator ran without errors.
Of 120 tables, all smaller tables
have the correct rowcount, but all
larger tables are 'truncated' at
around 10 million rows. I haven't
looked at table content.
8.3 8.4cvs
| rowcount | rowcount |
+-----------+----------+
| 7708808 | 7708808 |
| 10189986 | 9826926 |
| 2598808 | 2598808 |
| 6074119 | 6074119 |
| 13563559 | 9830128 |
| 13597969 | 9830114 |
| 23507096 | 9829964 |
| 15517779 | 9805163 |
| 12618858 | 9829974 |
| 11127521 | 9829902 |
| 4728763 | 4728763 |
| 4728763 | 4728763 |
| 61193699 | 9173790 |
| 1968328 | 1968328 |
| 53213115 | 9823165 |
| 31601320 | 9830060 |
One complication: I hadn't noticed
that there were 2 tables with a not yet
installed datatype. These tables were
simply not created by the pg_migrator-run.
I don't know how this influenced the results,
but I'll repeat it in the coming days.
Erik Rijkers
Erik Rijkers wrote:
2009.05.09
pg_migrator alpha 5 results
from PostgreSQL 8.3.7 to 8.4cvsCentos 5.2 x86_64 GNU/Linux
pg_migrator ran without errors.
Of 120 tables, all smaller tables
have the correct rowcount, but all
larger tables are 'truncated' at
around 10 million rows. I haven't
looked at table content.
Can you please show
SELECT relname, pg_relation_size(oid) FROM pg_class
for your tables?
I'm wondering that it could have forgotten to migrate the later table
segments ...
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
On Sun, May 10, 2009 02:05, Alvaro Herrera wrote:
Erik Rijkers wrote:
2009.05.09
pg_migrator alpha 5 results
from PostgreSQL 8.3.7 to 8.4cvsCentos 5.2 x86_64 GNU/Linux
pg_migrator ran without errors.
Of 120 tables, all smaller tables
have the correct rowcount, but all
larger tables are 'truncated' at
around 10 million rows. I haven't
looked at table content.Can you please show
SELECT relname, pg_relation_size(oid) FROM pg_class
for your tables?I'm wondering that it could have forgotten to migrate the later table
segments ...
In the following list, I have put
count(*) of each table next to the
pg_relation_size(oid) from pg_class:
It seems al 'truncated' tables give
pg_relation_size(oid) = 1073741824
(not the same subset as in previous mail)
| v 8.3.7 | 8.4 cvs head
| count(*) | pg_relation_size(oid) | count(*) | pg_relation_size(oid)
+----------+----------------------- ---+----------+----------------------
| 15 | 8192 | 15 | 8192
| 7708808 | 842334208 | 7708808 | 842334208
| 10189986 | 1113432064 | 9826926 | 1073741824
| 2598808 | 284360704 | 2598808 | 284360704
| 6074119 | 663576576 | 6074119 | 663576576
| 13563559 | 1481555968 | 9830128 | 1073741824
| 13597969 | 1485307904 | 9830114 | 1073741824
| 23507096 | 2567831552 | 9829964 | 1073741824
| 15517779 | 1699192832 | 9805163 | 1073741824
| 12618858 | 1378377728 | 9829974 | 1073741824
| 11127521 | 1215488000 | 9829902 | 1073741824
| 56 | 16384 | 56 | 16384
| 12667465 | 1278066688 | 10647510 | 1073741824
| 4728763 | 1623310336 | 4728763 | 553402368
| 4728763 | 553402368 | 4728763 | 553402368
| 10 | 8192 | 10 | 8192
| 100 | 16384 | 100 | 16384
| 13843420 | 1620082688 | 9175040 | 1073741824
| 1523927 | 215244800 | 1523927 | 215244800
| 1614576 | 219340800 | 1614576 | 219340800
| 61193699 | 7162437632 | 9173790 | 1073741824
| 1968328 | 215367680 | 1968328 | 215367680
| 53213115 | 5817516032 | 9823165 | 1073741824
| 31462058 | 3440214016 | 9821425 | 1073741824
| 31601320 | 3451863040 | 9830060 | 1073741824
hth,
Erik Rijkers
Import Notes
Resolved by subject fallback
"Erik Rijkers" <er@xs4all.nl> writes:
On Sun, May 10, 2009 02:05, Alvaro Herrera wrote:
I'm wondering that it could have forgotten to migrate the later table
segments ...
It seems al 'truncated' tables give
pg_relation_size(oid) = 1073741824
Looks like Alvaro nailed it. Bruce, you reading?
regards, tom lane
Tom Lane wrote:
"Erik Rijkers" <er@xs4all.nl> writes:
On Sun, May 10, 2009 02:05, Alvaro Herrera wrote:
I'm wondering that it could have forgotten to migrate the later table
segments ...It seems al 'truncated' tables give
pg_relation_size(oid) = 1073741824Looks like Alvaro nailed it. Bruce, you reading?
Yes, will research; thanks. I know we have multi-segment code, but
obviously it is not working reliably.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote:
Tom Lane wrote:
"Erik Rijkers" <er@xs4all.nl> writes:
On Sun, May 10, 2009 02:05, Alvaro Herrera wrote:
I'm wondering that it could have forgotten to migrate the later table
segments ...It seems al 'truncated' tables give
pg_relation_size(oid) = 1073741824Looks like Alvaro nailed it. Bruce, you reading?
Yes, will research; thanks. I know we have multi-segment code, but
obviously it is not working reliably.
I have fixed pg_migrator's multi-segment code and have released an
alpha-6 that fixes all known bugs:
http://pgfoundry.org/frs/?group_id=1000235
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Erik Rijkers wrote:
One complication: I hadn't noticed
that there were 2 tables with a not yet
installed datatype. These tables were
simply not created by the pg_migrator-run.I don't know how this influenced the results,
but I'll repeat it in the coming days.
I have modified pg_migrator so that an error on schema restore will be
displayed to the user and pg_migrator will exit with an error.
FYI, the multi-segment bug you reported is fixed in a new alpha-6
release too.
Thanks for the testing and the report.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +