Question on pg_cron
Hello All,
We have around 10 different partition tables for which the partition
maintenance is done using pg_partman extension. These tables have foreign
key dependency between them. We just called partman.run_maintanance_proc()
through pg_cron without any parameters and it was working fine. So we can
see only one entry in the cron.job table. And it runs daily once.
It was all working fine and we were seeing the historical partition being
dropped and new partitions being created without any issue. But suddenly we
started seeing, its getting failed with error "ERROR: can not drop
schema1.tab1_part_p2023_12_01 because other objects depend on it"
Then we realized , it may be the case that it's trying to run the partition
maintenance for the Parent partition table first before the child partition
table. So not sure how pg_partman handles the ordering of tables while
doing the partition maintenance as we don't see any parameter to drive the
ordering of the partition maintenance in part_config and we were under the
impression pg_partman will take care of the sequence of partition
maintenance automatically.
So want to understand if anybody encountered such issues?
And to handle the above issue , we are planning to call the partition
maintenance of each of the TABLE by passing the table name explicitly to
the run_maintanance_proc(), something as below. Is this advisable?
SELECT cron.unschedule('run_maintenance_proc');
SELECT cron.schedule(
'daily_partition_maintenance',
'0 2 * * *',
$$
DO $$
BEGIN
-- Run maintenance for child tables first
PERFORM partman.run_maintenance_proc('schema1.child_table1');
PERFORM partman.run_maintenance_proc('schema1.child_table2');
-- Add more child tables as needed
-- Run maintenance for parent tables next
PERFORM partman.run_maintenance_proc('schema1.parent_table1');
PERFORM partman.run_maintenance_proc('schema1.parent_table2');
-- Add more parent tables as needed
END;
$$;
$$
);
*Or else *
create a function like below and then call/schedule it through pg_cron
CREATE OR REPLACE FUNCTION run_partition_maintenance()
RETURNS void AS $$
BEGIN
-- Run maintenance for child tables first
PERFORM partman.run_maintenance_proc('schema1.child_table1');
PERFORM partman.run_maintenance_proc('schema1.child_table2');
-- Add more child tables as needed
-- Run maintenance for parent tables next
PERFORM partman.run_maintenance_proc('schema1.parent_table1');
PERFORM partman.run_maintenance_proc('schema1.parent_table2');
-- Add more parent tables as needed
END;
$$ LANGUAGE plpgsql;
SELECT cron.unschedule('run_maintenance_proc');
SELECT cron.schedule(
'daily_partition_maintenance',
'0 2 * * *',
'CALL run_partition_maintenance()'
);
On Sat, Jun 8, 2024 at 5:31 AM yudhi s <learnerdatabase99@gmail.com> wrote:
Hello All,
We have around 10 different partition tables for which the partition
maintenance is done using pg_partman extension. These tables have foreign
key dependency between them. We just called partman.run_maintanance_proc()
through pg_cron without any parameters and it was working fine. So we can
see only one entry in the cron.job table. And it runs daily once.It was all working fine and we were seeing the historical partition being
dropped and new partitions being created without any issue. But suddenly we
started seeing, its getting failed with error "ERROR: can not drop
schema1.tab1_part_p2023_12_01 because other objects depend on it"
Have you changed version lately of PG, pg_cron or pg_partman? Or maybe
what pg_cron or pg_partman depends on?
On Sat, 8 Jun, 2024, 9:53 pm Ron Johnson, <ronljohnsonjr@gmail.com> wrote:
On Sat, Jun 8, 2024 at 5:31 AM yudhi s <learnerdatabase99@gmail.com>
wrote:Hello All,
We have around 10 different partition tables for which the partition
maintenance is done using pg_partman extension. These tables have foreign
key dependency between them. We just called partman.run_maintanance_proc()
through pg_cron without any parameters and it was working fine. So we can
see only one entry in the cron.job table. And it runs daily once.It was all working fine and we were seeing the historical partition being
dropped and new partitions being created without any issue. But suddenly we
started seeing, its getting failed with error "ERROR: can not drop
schema1.tab1_part_p2023_12_01 because other objects depend on it"Have you changed version lately of PG, pg_cron or pg_partman? Or maybe
what pg_cron or pg_partman depends on?
No version change, but we updated the part_config to set premake from 30 to
60 for all the tables. But not sure how that impacted this behavior.
However, do you think, we should better control the order of execution
rather letting postgres to decide it's own, considering there is no such
parameters for this ordering in part_config? And in that case which
approach should we use out of the two i mentioned. Or any other strategies,
should we follow, please advise?
Show quoted text
On Sat, Jun 8, 2024 at 10:05 PM yudhi s <learnerdatabase99@gmail.com> wrote:
On Sat, 8 Jun, 2024, 9:53 pm Ron Johnson, <ronljohnsonjr@gmail.com> wrote:
On Sat, Jun 8, 2024 at 5:31 AM yudhi s <learnerdatabase99@gmail.com>
wrote:Hello All,
We have around 10 different partition tables for which the partition
maintenance is done using pg_partman extension. These tables have foreign
key dependency between them. We just called partman.run_maintanance_proc()
through pg_cron without any parameters and it was working fine. So we can
see only one entry in the cron.job table. And it runs daily once.It was all working fine and we were seeing the historical partition
being dropped and new partitions being created without any issue. But
suddenly we started seeing, its getting failed with error "ERROR: can not
drop schema1.tab1_part_p2023_12_01 because other objects depend on it"Have you changed version lately of PG, pg_cron or pg_partman? Or maybe
what pg_cron or pg_partman depends on?No version change, but we updated the part_config to set premake from 30
to 60 for all the tables. But not sure how that impacted this behavior.However, do you think, we should better control the order of execution
rather letting postgres to decide it's own, considering there is no such
parameters for this ordering in part_config? And in that case which
approach should we use out of the two i mentioned. Or any other strategies,
should we follow, please advise?
I believe, You should log this as an issue in the pg_partman open source
project.