Additive backup and restore?
I am designing a database for a web application. In the near future I will
require past data for Audit, Security and Analysis purpose. But storing all
that data will increase the database size. What are some well known
approaches to archiving data for later use without increasing the database
size?
One approach I am thinking of is compressed additive backup and restore
i.e. copy the rows of the tables that will be required later and store them
in a compressed form, and then delete those rows from the table. And when
the data is required then restore them from the backup files in an additive
way.
Is there an easy way to achieve this, say via pg_dump?
Consider table partitioning. You can detach, save and delete partitions, and you can restore and attach partitions.
Show quoted text
On 8 May 2023 12:24:06 CEST, Age Apache <age.apache@gmail.com> wrote:
I am designing a database for a web application. In the near future I will
require past data for Audit, Security and Analysis purpose. But storing all
that data will increase the database size. What are some well known
approaches to archiving data for later use without increasing the database
size?One approach I am thinking of is compressed additive backup and restore
i.e. copy the rows of the tables that will be required later and store them
in a compressed form, and then delete those rows from the table. And when
the data is required then restore them from the backup files in an additive
way.Is there an easy way to achieve this, say via pg_dump?
On 5/8/23 05:24, Age Apache wrote:
I am designing a database for a web application. In the near future I will
require past data for Audit, Security and Analysis purpose. But storing
all that data will increase the database size. What are some well known
approaches to archiving data for later use without increasing the database
size?One approach I am thinking of is compressed additive backup
Additive?
Why not one compressed archive file per month per table?
and restore i.e. copy the rows of the tables that will be required later
and store them in a compressed form, and then delete those rows from the
table. And when the data is required then restore them from the backup
files in an additive way.Is there an easy way to achieve this, say via pg_dump?
If the tables are partitioned on the relevant date field, then archiving old
data will be "trivially" easy.
If not, then it'll still be "easy":
- COPY TO a file
- compress it
- store it somewhere.
Reverse to "de-archive" the data.
--
Born in Arizona, moved to Babylonia.