Patch for migration of the pg_commit_ts directory

Started by ls77778 months ago23 messages
#1ls7777
ls7777
ls7777@yandex.ru
1 attachment(s)

Attachments:

v1-0001-Migration-of-the-pg_commit_ts-directory.patchtext/x-diff; name=v1-0001-Migration-of-the-pg_commit_ts-directory.patch
#2Maxim Orlov
Maxim Orlov
orlovmg@gmail.com
In reply to: ls7777 (#1)
1 attachment(s)
Re: Patch for migration of the pg_commit_ts directory

Yes, the pg_commit_ts directory is not transferred by pg_upgrade.
Basically, you can get the exact same result by manually copying the
pg_commit_ts directory and running pg_resetwal. And this patch does it
automatically.

In other words, the patch provides the stated functionality, but
consider running pgindet.

I'm not sure whether there were any reasons not to make this move from
the beginning. The authors' opinions are needed here.

And one more thing. In my perspective, having some tests is always a
good idea, even if they are not meant to be committed. After briefly
examining the patch, I developed this basic test. This is not finalized
in any manner, but it is simply an idea.

--
Best regards,
Maxim Orlov.

Attachments:

006_commit_ts.pl.txttext/plain; charset=US-ASCII; name=006_commit_ts.pl.txt
#3Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#1)
RE: Patch for migration of the pg_commit_ts directory

Hi,

Per my understanding, track_commit_timestamp must be on the new instance,
otherwise it would be removed when new instance starts. Is it correct?
If so, should we detect the difference and do some warnings/errors here?
Or it is just the user's responsibility?

Best regards,
Hayato Kuroda
FUJITSU LIMITED

#4Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: Maxim Orlov (#2)
RE: Patch for migration of the pg_commit_ts directory

Dear Maxim,

Thanks for considering the test case. To confirm, where should we put the test code?
Pp_upgrade has already had 006 test, and commit_ts has not had 005 test.

Regarding the test, it is enough to check whether oldest/newest transactions on the old
cluster can be tracked.

Best regards,
Hayato Kuroda
FUJITSU LIMITED

#5ls7777
ls7777
ls7777@yandex.ru
In reply to: Hayato Kuroda (Fujitsu) (#3)
Re: Patch for migration of the pg_commit_ts directory

<div><div>Hi,</div><div> </div><div>Yes, track_commit_timestamp must be installed in the new instance.</div><div>This is only the responsibility of an experienced user.</div><div>pg_upgrage should allow you to save pg_commit_ts if this data exists at the time of migration.</div></div><div><div>Warnings are not needed, the loss of this data is not critical in most cases. They were lost with each migration if users did not manually migrate them.</div><div> </div></div><div>----------------</div><div>Кому: 'ls7777' (ls7777@yandex.ru);</div><div>Копия: pgsql-hackers@postgresql.org;</div><div>Тема: Patch for migration of the pg_commit_ts directory;</div><div>01.10.2025, 17:17, "Hayato Kuroda (Fujitsu)" &lt;kuroda.hayato@fujitsu.com&gt;:</div><blockquote><p>Hi,<br /><br />Per my understanding, track_commit_timestamp must be on the new instance,<br />otherwise it would be removed when new instance starts. Is it correct?<br />If so, should we detect the difference and do some warnings/errors here?<br />Or it is just the user's responsibility?<br /><br />Best regards,<br />Hayato Kuroda<br />FUJITSU LIMITED<br /> </p></blockquote>

#6Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#5)
RE: Patch for migration of the pg_commit_ts directory

Hi,
(Sorry but I cannot find your name)

Yes, track_commit_timestamp must be installed in the new instance.
This is only the responsibility of an experienced user.
pg_upgrage should allow you to save pg_commit_ts if this data exists at the time of migration.
Warnings are not needed, the loss of this data is not critical in most cases.
They were lost with each migration if users did not manually migrate them.

So, your policy is that commit_ts is not the user data thus it is OK to drop during
the upgrade, is it correct? I want to know other's opinion around here.
Note that if we want to check, it can be done in check_control_data().

Regarding the patch:
```
-
+	cluster->controldata.chkpnt_oldstCommitTsxid = 0;
+	cluster->controldata.chkpnt_newstCommitTsxid = 0;
```

Other attributes are not initialized, you can follow.

```
+	if (old_cluster.controldata.chkpnt_oldstCommitTsxid > 0)
+					   copy_subdir_files("pg_commit_ts", "pg_commit_ts");
```

Indent should be fixed. Please run pgindent.

```
-			  old_cluster.controldata.chkpnt_nxtxid,
-			  old_cluster.controldata.chkpnt_nxtxid,
+			  old_cluster.controldata.chkpnt_oldstCommitTsxid == 0 ? old_cluster.controldata.chkpnt_nxtxid : old_cluster.controldata.chkpnt_oldstCommitTsxid,
+			  old_cluster.controldata.chkpnt_newstCommitTsxid == 0 ? old_cluster.controldata.chkpnt_nxtxid : old_cluster.controldata.chkpnt_newstCommitTsxid,
```

To confirm, is there a possibility that only either of oldest/newest CommitTs exists?

Best regards,
Hayato Kuroda
FUJITSU LIMITED

#7Amit Kapila
Amit Kapila
amit.kapila16@gmail.com
In reply to: Hayato Kuroda (Fujitsu) (#6)
Re: Patch for migration of the pg_commit_ts directory

On Thu, Oct 2, 2025 at 12:10 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:

Yes, track_commit_timestamp must be installed in the new instance.
This is only the responsibility of an experienced user.
pg_upgrage should allow you to save pg_commit_ts if this data exists at the time of migration.
Warnings are not needed, the loss of this data is not critical in most cases.
They were lost with each migration if users did not manually migrate them.

So, your policy is that commit_ts is not the user data thus it is OK to drop during
the upgrade, is it correct?

IIUC, the proposal is that if GUC track_commit_timestamp is enabled on
the new instance then we should copy it, otherwise, we can drop
copying it. Is my understanding correct? I think we can follow what is
done check_new_cluster_replication_slots() for the case when
track_commit_timestamp is not set on the new server. When we try to
copy slots and the wal_level on the new server is minimal, we error
out, so shouldn't we do the same here and error_out if
track_commit_timestamp is not enabled and we have some valid commit_ts
data to copy?

--
With Regards,
Amit Kapila.

#8Maxim Orlov
Maxim Orlov
orlovmg@gmail.com
In reply to: Amit Kapila (#7)
Re: Patch for migration of the pg_commit_ts directory

On Thu, 2 Oct 2025 at 11:49, Amit Kapila <amit.kapila16@gmail.com> wrote:

When we try to
copy slots and the wal_level on the new server is minimal, we error
out, so shouldn't we do the same here and error_out if
track_commit_timestamp is not enabled and we have some valid commit_ts
data to copy?

+1 Sounds reasonable to me. It's better to give an explicit error;
otherwise, we would remove data that the user clearly wants to migrate
to the new cluster. And deleting data just because the user forgot to
specify one parameter in a new cluster looks like a bad joke to me.

--
Best regards,
Maxim Orlov.

#9ls7777
ls7777
ls7777@yandex.ru
In reply to: Maxim Orlov (#8)
Re: Patch for migration of the pg_commit_ts directory

<div><div>Hi,</div><div><div><span style="white-space:pre-wrap">I'll try to return it to the original </span><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">thread.</span></div><div> </div><div><div>I absolutely agree with you. Data loss during migration is an unacceptable situation. And why it's still happening, I don't understand. But issuing messages will require translating it into other languages and complicate the patch.  I'm not a programmer to do it beautifully. And not a beautiful patch is unlikely to be accepted. </div><div>I will do a parameter check and an ERROR message.</div></div><div> </div></div><div> </div><blockquote><div><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Hi,</span><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Could you please reply to the original thread? Otherwise, it is difficult to track later.</span><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Regarding the patch, I think Amit suggested to raise an ERROR in case of parameter</span><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">mismatch, and Maxim agreed the point [2]. How do you feel?</span><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">[1]: </span><a href="/messages/by-id/CAA4eK1+Zayox=x84upet_gc4hrCKSwZSpnO9=Q4vH8rrJbsBOg@mail.gmail.com&quot; rel="noopener noreferrer" target="_blank" style="background-color:rgb( 255 , 255 , 255 );color:rgb( 0 , 119 , 255 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">/messages/by-id/CAA4eK1+Zayox=x84upet_gc4hrCKSwZSpnO9=Q4vH8rrJbsBOg@mail.gmail.com&lt;/a&gt;&lt;br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">[2]: </span><a href="/messages/by-id/CACG=ezaFdL4EM8bdrHTRVHt5BmxxWGXcPiDr6Qm00Qj+S_Y+vQ@mail.gmail.com&quot; rel="noopener noreferrer" target="_blank" style="background-color:rgb( 255 , 255 , 255 );color:rgb( 0 , 119 , 255 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">/messages/by-id/CACG=ezaFdL4EM8bdrHTRVHt5BmxxWGXcPiDr6Qm00Qj+S_Y+vQ@mail.gmail.com&lt;/a&gt;&lt;br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Best regards,</span><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Hayato Kuroda</span><br style="background-color:rgb( 255 , 255 , 255 );color:rgb( 26 , 26 , 26 );font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" /><span style="background-color:#ffffff;color:#1a1a1a;float:none;font-family:'ys text' , 'arial' , sans-serif;font-size:16px;font-style:normal;font-weight:400;text-align:start;text-decoration-color:initial;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">FUJITSU LIMITED</span></div></blockquote></div><div> </div><div>----------------</div><div>Кому: Amit Kapila (amit.kapila16@gmail.com);</div><div>Копия: Hayato Kuroda (Fujitsu) (kuroda.hayato@fujitsu.com), pgsql-hackers@postgresql.org;</div><div>Тема: Patch for migration of the pg_commit_ts directory;</div><div>06.10.2025, 12:59, "Maxim Orlov" &lt;orlovmg@gmail.com&gt;:</div><blockquote><div><div> </div> <div><div>On Thu, 2 Oct 2025 at 11:49, Amit Kapila &lt;<a href="mailto:amit.kapila16@gmail.com" rel="noopener noreferrer">amit.kapila16@gmail.com</a>&gt; wrote:</div><blockquote style="border-left-color:rgb( 204 , 204 , 204 );border-left-style:solid;border-left-width:1px;margin:0px 0px 0px 0.8ex;padding-left:1ex"> When we try to<br />copy slots and the wal_level on the new server is minimal, we error<br />out, so shouldn't we do the same here and error_out if<br />track_commit_timestamp is not enabled and we have some valid commit_ts<br />data to copy?</blockquote><div> </div><div>+1 Sounds reasonable to me. It's better to give an explicit error;<br />otherwise, we would remove data that the user clearly wants to migrate<br />to the new cluster. And deleting data just because the user forgot to<br />specify one parameter in a new cluster looks like a bad joke to me.   </div><div> </div></div>-- <div><div><div>Best regards,</div><div>Maxim Orlov.</div></div></div></div></blockquote>

#10Bruce Momjian
Bruce Momjian
bruce@momjian.us
In reply to: ls7777 (#9)
Re: Patch for migration of the pg_commit_ts directory

On Mon, Oct 6, 2025 at 07:31:52PM +0500, ls7777 wrote:

Hi,
I'll try to return it to the original thread.
 
I absolutely agree with you. Data loss during migration is an unacceptable
situation. And why it's still happening, I don't understand. But issuing
messages will require translating it into other languages and complicate the
patch.  I'm not a programmer to do it beautifully. And not a beautiful patch is
unlikely to be accepted. 
I will do a parameter check and an ERROR message.

You don't need to worry about translations. Each language team will
make sure your English message is translated.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

#11ls7777
ls7777
ls7777@yandex.ru
In reply to: Maxim Orlov (#8)
1 attachment(s)
Re: Patch for migration of the pg_commit_ts directory

Attachments:

v3-Migration-of-the-pg_commit_ts-directory.patchtext/x-diff; name=v3-Migration-of-the-pg_commit_ts-directory.patch
#12Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#11)
RE: Patch for migration of the pg_commit_ts directory

Hi,

Thanks for updating the patch.

Regarding the check_track_commit_timestamp_parameter(), I'm not sure the function
is needed. Since pg_controldata outputs whether the commit_ts module is enabled
or not [1]``` $ pg_controldata data/ | grep "track_" track_commit_timestamp setting: off ```, can we obtain there instead?
I imagined like we can add a new field at ControlData, it could be obtained in get_control_data(),
values for instances could be compared in check_control_data().

Another possibility is that oldestCommitTsXid/newestCommitTsXid can be always
valid if track_commit_timestamp is on. In this case the attribute is not needed
anymore. It is enough to check these values. Can you check them?

Regarding the test, I feel it is unnecessary long. I feel it is enough to ensure
one or two transactions have the same timestamp on the both nodes, pgbench is not
needed. Current test may not be able to finish on the slow machine.

[1]: ``` $ pg_controldata data/ | grep "track_" track_commit_timestamp setting: off ```
```
$ pg_controldata data/ | grep "track_"
track_commit_timestamp setting: off
```

Best regards,
Hayato Kuroda
FUJITSU LIMITED

#13ls7777
ls7777
ls7777@yandex.ru
In reply to: Hayato Kuroda (Fujitsu) (#12)
Re: Patch for migration of the pg_commit_ts directory

<div>Hi,</div><div> </div><div><div>At the time check_control_data is launched for the new cluster, the control file does not contain information about the set track_commit_timestamp=on. It is installed only in postgresql.conf. The check_track_commit_timestamp_parameter function is only needed for the new cluster. And you can not run it for the old cluster, but use data from the old_cluster.controldata.chkpnt_newstCommitTsxid control file. But this will be less clear than using the check_track_commit_timestamp_parameter function.</div><div> </div><div>The test can be shortened.</div></div><div> </div><div>----------------</div><div>Кому: 'ls7777' (ls7777@yandex.ru), orlovmg@gmail.com (orlovmg@gmail.com), amit.kapila16@gmail.com (amit.kapila16@gmail.com);</div><div>Копия: pgsql-hackers@postgresql.org;</div><div>Тема: Patch for migration of the pg_commit_ts directory;</div><div>07.10.2025, 08:15, "Hayato Kuroda (Fujitsu)" &lt;kuroda.hayato@fujitsu.com&gt;:</div><blockquote><p>Hi,<br /><br />Thanks for updating the patch.<br /><br />Regarding the check_track_commit_timestamp_parameter(), I'm not sure the function<br />is needed. Since pg_controldata outputs whether the commit_ts module is enabled<br />or not [1], can we obtain there instead?<br />I imagined like we can add a new field at ControlData, it could be obtained in get_control_data(),<br />values for instances could be compared in check_control_data().<br /><br />Another possibility is that oldestCommitTsXid/newestCommitTsXid can be always<br />valid if track_commit_timestamp is on. In this case the attribute is not needed<br />anymore. It is enough to check these values. Can you check them?<br /><br />Regarding the test, I feel it is unnecessary long. I feel it is enough to ensure<br />one or two transactions have the same timestamp on the both nodes, pgbench is not<br />needed. Current test may not be able to finish on the slow machine.<br /><br />[1]<br />```<br />$ pg_controldata data/ | grep "track_"<br />track_commit_timestamp setting: off<br />```<br /><br />Best regards,<br />Hayato Kuroda<br />FUJITSU LIMITED<br /> </p></blockquote>

#14Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#13)
RE: Patch for migration of the pg_commit_ts directory

Hi,

At the time check_control_data is launched for the new cluster, the control file
does not contain information about the set track_commit_timestamp=on. It is
installed only in postgresql.conf.

You did consider the case that track_commit_timestamp is set to on but the instance
has not started before the pg_upgrade, right? Valid point.

The check_track_commit_timestamp_parameter function is only needed for the new
cluster. And you can not run it for the old cluster, but use data from the
old_cluster.controldata.chkpnt_newstCommitTsxid control file. But this will be
less clear than using the check_track_commit_timestamp_parameter function.

Sorry, I'm not very clear this part. Can you describe the point bit more?

Other comments:

01. get_control_data

```
+	if (GET_MAJOR_VERSION(cluster->major_version) < 905)
+	{
+		cluster->controldata.chkpnt_oldstCommitTsxid = 0;
+		cluster->controldata.chkpnt_newstCommitTsxid = 0;
+	}
```

IIUC, checksum_version is checked like this because got_data_checksum_version
must be also set.
Since we do not have the boolean for commit_ts, not sure it is needed.
old_cluster and new_cluster are the global variable and they are initialized with
zero.

02. check_track_commit_timestamp_parameter

```
+		conn = connectToServer(cluster, "template1");
+		res = executeQueryOrDie(conn, "SELECT count(*) AS is_set "
+								"FROM pg_settings WHERE name = 'track_commit_timestamp' and setting = 'on'");
+		is_set = PQfnumber(res, "is_set");
+		cluster->track_commit_timestamp_on = atoi(PQgetvalue(res, 0, is_set)) == 1;
```

The SQL looks strange for me. Isn't it enough to directly obtain "setting" attribute
and check it is "on" or not? Also, conn and res can be the local variable of the
else part.

03. ClusterInfo

```
+	bool		track_commit_timestamp_on;	/* track_commit_timestamp for
+										 *    cluster */
```

The indent is broken, please run pgindent.

04. test

Could you please update meson.build? Otherwise the added test could not be run for meson build.

Best regards,
Hayato Kuroda
FUJITSU LIMITED
 

#15ls7777
ls7777
ls7777@yandex.ru
In reply to: Hayato Kuroda (Fujitsu) (#14)
1 attachment(s)
Re: Patch for migration of the pg_commit_ts directory

Attachments:

v4-Migration-of-the-pg_commit_ts-directory.patchtext/x-diff; name=v4-Migration-of-the-pg_commit_ts-directory.patch
#16Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#15)
RE: Patch for migration of the pg_commit_ts directory

Hi,

Thanks for updating the patch and sorry for the late reply. Here are my comments.

01.
```
+ prep_status("Checking for pg_commit_ts");
```

I think we must clarify which node is being checked. Something like:
Checking for new cluster configuration for commit timestamp

02.
```
}
-
/* we have the result of cmd in "output". so parse it line by line now */
```

This change is not needed.

03.
```
+	/*
+	 * Copy pg_commit_ts
+	 */
```

I feel it can be removed or have more meanings. Something lile:
Copy old commit_timestamp data to new, if available.

04.

Regarding the test,

05.
```
+sub command_output
```

Can run_command() be used instead of defining new function?

06.
```
+$old->command_ok([ 'pgbench', '-i', '-s', '1' ], 'init pgbench');
+$old->command_ok([ 'pgbench', '-t', '1', '-j', '1' ], 'pgbench it');
```

I think no need to use pgbench anymore. Can we define tables and insert tuples
by myself?

07.
```
+command_fails(
+	[
+		'pg_upgrade', '--no-sync',
+		'-d', $old->data_dir,
+		'-D', $new->data_dir,
+		'-b', $old->config_data('--bindir'),
+		'-B', $new->config_data('--bindir'),
+		'-s', $new->host,
+		'-p', $old->port,
+		'-P', $new->port,
+		$mode
+	],
+	'run of pg_upgrade fails with mismatch parameter track_commit_timestamp');
```

According to other test files, we do not use the shorten option.
Also, please verify the actual output by command_ok_or_fails_like() or command_checks_all().

08.
```
+sub xact_commit_ts
```

Not sure, but this function is introduced because we have 100 transactions. Can we omit this now?

09.
```
+# The new cluster should contain timestamps created during the pg_upgrade and
+# some more created by the pgbench.
+#
+print "\nCommit timestamp only in new cluster:\n";
+for my $line (@b) {
+	print "$line\n" unless $h1{$line};
+}
```

I don't think this is needed because the output is not verified.

Best regards,
Hayato Kuroda
FUJITSU LIMITED

#17ls7777
ls7777
ls7777@yandex.ru
In reply to: Hayato Kuroda (Fujitsu) (#16)
1 attachment(s)
Re: Patch for migration of the pg_commit_ts directory

Attachments:

v5-Migration-of-the-pg_commit_ts-directory.patchtext/x-diff; name=v5-Migration-of-the-pg_commit_ts-directory.patch
#18Maxim Orlov
Maxim Orlov
orlovmg@gmail.com
In reply to: ls7777 (#17)
Re: Patch for migration of the pg_commit_ts directory

Looks good to me. As I see it, the patch is almost ready, so I decided
to create an CF entry [0]https://commitfest.postgresql.org/patch/6119/. Feel free to edit it as you see fit.

And your mail also includes attachment (unknown_filename
</messages/by-id/attachment/175614/unknown_filename&gt;
), interfering with the CF-bot.

[0]: https://commitfest.postgresql.org/patch/6119/

#19ls7777
ls7777
ls7777@yandex.ru
In reply to: Maxim Orlov (#18)
Re: Patch for migration of the pg_commit_ts directory

<div><div>Thanks.</div><div>That's the good news.</div><div><div><div><span style="white-space:pre-wrap">Thanks to Hayato Kuroda, for the advice.</span></div><div><span style="white-space:pre-wrap">Thanks to everyone who participated in the promotion of the patch.</span></div><div> </div></div></div></div><div> </div><div>----------------</div><div>Кому: ls7777 (ls7777@yandex.ru);</div><div>Копия: Hayato Kuroda (Fujitsu) (kuroda.hayato@fujitsu.com), pgsql-hackers@postgresql.org, amit.kapila16@gmail.com;</div><div>Тема: Patch for migration of the pg_commit_ts directory;</div><div>10.10.2025, 15:55, "Maxim Orlov" &lt;orlovmg@gmail.com&gt;:</div><blockquote><div><div><span lang="en">Looks good to me. As I see it, the patch is almost ready, so I decided </span></div><div><span lang="en">to create an CF entry [0]. Feel free to edit it as you see fit.</span></div><div> </div><div><span lang="en">And your mail also includes a</span>ttachment (<a href="/messages/by-id/attachment/175614/unknown_filename&quot; rel="noopener noreferrer">unknown_filename</a>), <span lang="en">interfering with the CF-bot.</span></div><div> </div><div>[0] <a href="https://commitfest.postgresql.org/patch/6119/&quot; rel="noopener noreferrer">https://commitfest.postgresql.org/patch/6119/&lt;/a&gt;&lt;/div&gt;&lt;/div&gt;&lt;/blockquote&gt;

#20Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#17)
1 attachment(s)
RE: Patch for migration of the pg_commit_ts directory

Hi,

Thanks for updating the patch.

```
+my ($xid,$ts) = $resold =~ /\s*(\d+)\s*\|(.*)/;
```

Per my understanding $ts is not used here. Can you remove it?

Apart from above, I found some cosmetic issues. Please see attached
my fix which can be applied atop HEAD. Can you check and include if
it is OK?

Best regards,
Hayato Kuroda
FUJITSU LIMITED
 

Attachments:

kuroda.diffsapplication/octet-stream; name=kuroda.diffs
#21ls7777
ls7777
ls7777@yandex.ru
In reply to: ls7777 (#1)
1 attachment(s)
Re: Patch for migration of the pg_commit_ts directory

Attachments:

v6-Migration-of-the-pg_commit_ts-directory.patchtext/x-diff; name=v6-Migration-of-the-pg_commit_ts-directory.patch
#22Hayato Kuroda (Fujitsu)
Hayato Kuroda (Fujitsu)
kuroda.hayato@fujitsu.com
In reply to: ls7777 (#21)
1 attachment(s)
RE: Patch for migration of the pg_commit_ts directory

Hi,

Thanks for updating the patch. Mostly looks good.

I ran pgindent locally and indents of my part were incorrect. PSA the
included version.

I have no comments anymore. Thanks for the great work.

Best regards,
Hayato Kuroda
FUJITSU LIMITED

Attachments:

v7-0001-Migration-of-the-pg_commit_ts-directory.patchapplication/octet-stream; name=v7-0001-Migration-of-the-pg_commit_ts-directory.patch
#23Maxim Orlov
Maxim Orlov
orlovmg@gmail.com
In reply to: Hayato Kuroda (Fujitsu) (#22)
Re: Patch for migration of the pg_commit_ts directory

On Wed, 15 Oct 2025 at 04:46, Hayato Kuroda (Fujitsu) <
kuroda.hayato@fujitsu.com> wrote:

Hi,

Thanks for updating the patch. Mostly looks good.

Yep, looks good to me too.

If there are no objections, I will move forward to mark the thread as
ready for committers.

Hayato Kuroda, you did a great job reviewing this patch; consider to
tag yourself as a reviewer [0]https://commitfest.postgresql.org/patch/6119/.

[0]: https://commitfest.postgresql.org/patch/6119/

--
Best regards,
Maxim Orlov.