PATCH: Exclude temp relations from base backup

Hi,

On 2/28/18 10:55 AM, David Steele wrote:

This is a follow-up patch from the exclude unlogged relations discussion
[1].

The patch excludes temporary relations during a base backup using the
existing looks_like_temp_rel_name() function for identification.

It shares code to identify database directories from [1], so for now
that has been duplicated in this patch to make it independent. I'll
rebase depending on what gets committed first.

Updated the patch to change die() to BAIL_OUT() and use append_to_file()
as suggested for another test patch [1]/messages/by-id/6bc5d931-5b00-279f-f65a-26e32de400a6@pgmasters.net.

Regards,
--
-David
david@pgmasters.net

[1]: /messages/by-id/6bc5d931-5b00-279f-f65a-26e32de400a6@pgmasters.net
/messages/by-id/6bc5d931-5b00-279f-f65a-26e32de400a6@pgmasters.net

On 3/13/18 12:34 PM, David Steele wrote:

Updated the patch to change die() to BAIL_OUT() and use append_to_file()
as suggested for another test patch.

Updated patch now that the unlogged table exclusions have been committed
[1]: /messages/by-id/4d9be1c0-5c58-d9a0-7152-2771224910ae@sigaev.ru

Thanks,
--
-David
david@pgmasters.net

[1]: /messages/by-id/4d9be1c0-5c58-d9a0-7152-2771224910ae@sigaev.ru
/messages/by-id/4d9be1c0-5c58-d9a0-7152-2771224910ae@sigaev.ru

Hi!

Will autovacuum (or something else) complain about absense of relfile during
orphan table deleting? I mean, you get a base backup without temp tables, then
you try to run postgres on it and will it complain about existing record in
pg_class and absence of corresponding relfile?

David Steele wrote:

On 3/13/18 12:34 PM, David Steele wrote:

Updated the patch to change die() to BAIL_OUT() and use append_to_file()
as suggested for another test patch.

Updated patch now that the unlogged table exclusions have been committed
[1].

Thanks,

--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/

Greetings,

* Teodor Sigaev (teodor@sigaev.ru) wrote:

Will autovacuum (or something else) complain about absense of relfile during
orphan table deleting? I mean, you get a base backup without temp tables,
then you try to run postgres on it and will it complain about existing
record in pg_class and absence of corresponding relfile?

I would certainly hope not considering that's what happens during
regular crash recovery also, so if there's an issue with that, we'd have
a problem in released versions.

There's an independent discussion that was being had recently about how
to make sure those records in pg_class get cleaned up in a reasonable
timeframe and don't lead to problems with wrap-arounds, but that's a
different and pre-existing issue.

Thanks!

Stephen

On 3/26/18 1:06 PM, Stephen Frost wrote:

* Teodor Sigaev (teodor@sigaev.ru) wrote:

Will autovacuum (or something else) complain about absense of relfile during
orphan table deleting? I mean, you get a base backup without temp tables,
then you try to run postgres on it and will it complain about existing
record in pg_class and absence of corresponding relfile?

I would certainly hope not considering that's what happens during
regular crash recovery also, so if there's an issue with that, we'd have
a problem in released versions.

Agreed. The logic for pg_basebackup was modeled off RemovePgTempFiles()
which is called at postmaster start. We are just doing the cleanup in
advance (in the backup only, of course).

Thanks,
--
-David
david@pgmasters.net

Thank you, pushed

David Steele wrote:

On 3/26/18 1:06 PM, Stephen Frost wrote:

* Teodor Sigaev (teodor@sigaev.ru) wrote:

Will autovacuum (or something else) complain about absense of relfile during
orphan table deleting? I mean, you get a base backup without temp tables,
then you try to run postgres on it and will it complain about existing
record in pg_class and absence of corresponding relfile?

I would certainly hope not considering that's what happens during
regular crash recovery also, so if there's an issue with that, we'd have
a problem in released versions.

Agreed. The logic for pg_basebackup was modeled off RemovePgTempFiles()
which is called at postmaster start. We are just doing the cleanup in
advance (in the backup only, of course).

Thanks,

--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/