Make a 100_bugs.pl test more faster.

Started by Anton A. Melnikovabout 3 years ago2 messages
#1Anton A. Melnikov
aamelnikov@inbox.ru
2 attachment(s)

Hello!

The previous discussion was here:
/messages/by-id/b570c367-ba38-95f3-f62d-5f59b9808226@inbox.ru

On 15.11.2022 04:59, Tom Lane wrote:

"Anton A. Melnikov" <aamelnikov@inbox.ru> writes:

Additionally
i've tried to reduce overall number of nodes previously
used in this test in a similar way.

Optimizing existing tests isn't an answer to that. We could
install those optimizations without adding a new test case.

Here is a separate patch for the node usage optimization mentioned above.
It decreases the CPU usage during 100_bugs.pl by about 30%.

There are also some experimental data: 100_bugs-CPU-usage.txt

Would be glad for any comments and concerns.

With best regards,

--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

v1-0001-Reuse-nodes-in-100_bugs-test.patchtext/x-patch; charset=UTF-8; name=v1-0001-Reuse-nodes-in-100_bugs-test.patchDownload
commit 72ea85b5f6a64f2c53e6a55455d134b228b6994b
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Mon Nov 14 08:30:26 2022 +0300

    Reuse nodes in subscription/t/100_bugs.pl test to make in faster.

diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 6247aa7730..525584b2b6 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -81,9 +81,8 @@ $node_subscriber->stop('fast');
 # identity set before accepting updates.  If it did not it would cause
 # an error when an update was attempted.
 
-$node_publisher = PostgreSQL::Test::Cluster->new('publisher2');
-$node_publisher->init(allows_streaming => 'logical');
-$node_publisher->start;
+$node_publisher->rotate_logfile();
+$node_publisher->start();
 
 $node_publisher->safe_psql('postgres',
 	"CREATE PUBLICATION pub FOR ALL TABLES");
@@ -226,13 +225,13 @@ $node_sub->stop('fast');
 # target table's relcache was not being invalidated. This leads to skipping
 # UPDATE/DELETE operations during apply on the subscriber side as the columns
 # required to search corresponding rows won't get logged.
-$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
-$node_publisher->init(allows_streaming => 'logical');
-$node_publisher->start;
 
-$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
-$node_subscriber->init(allows_streaming => 'logical');
-$node_subscriber->start;
+$node_publisher->rotate_logfile();
+$node_publisher->start();
+
+$node_subscriber->rotate_logfile();
+$node_subscriber->rotate_logfile(); # call twice to synchronize lognames between master and replica
+$node_subscriber->start();
 
 $node_publisher->safe_psql('postgres',
 	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
100_bugs-CPU-usage.txttext/plain; charset=UTF-8; name=100_bugs-CPU-usage.txtDownload
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Anton A. Melnikov (#1)
Re: Make a 100_bugs.pl test more faster.

"Anton A. Melnikov" <aamelnikov@inbox.ru> writes:

Here is a separate patch for the node usage optimization mentioned above.
It decreases the CPU usage during 100_bugs.pl by about 30%.

Hmm ... as written, this isn't testing the same thing, because you
didn't disable the FOR ALL TABLES publications created in the earlier
steps, so we're redundantly syncing more publications in the later
ones. Cleaning those up seems to make it a little faster yet,
so pushed with that adjustment.

regards, tom lane