How to rebuild index efficiently
Hi Team,
i have a table of size 2.6TB which is very prone to updates and inserts so
we have tuned autovacuum to run on it very aggressively , so the table
level bloat is fine .
Now we are facing issue with indexes on this table. the total size of all
indexes on this table is around 2.4TB.
There is an unique index of 1.2TB size out of which 850 GB is bloat ,
creating another index concurrently is taking lot of time and we cannot
offer down time to reindex this index.
Could you please suggest any ideal approach to tackle this ?
Postgres version:9.6.15
postgres=> \dt+ cust_table
List of relations
Schema | Name | Type | Owner | Size | Description
--------+-----------------+-------+--------+---------+-------------
public | cust_table | table | raj | 2685 GB |
(1 row)
Index bloat:
database_name | schema_name | table_name |
index_name | bloat_pct | bloat_mb | index_mb | table_mb |
index_scans
---------------+-------------+-------------------------------+---------------------------------+-----------+----------+-------------+-------------+-------------
postgres | public | cust_table |
unique_event_type_action_id | 69 | 884477 | 1285743.648 |
2749094.070 | 342466359
postgres=> \di+ unique_event_type_action_id
List of relations
Schema | Name | Type | Owner | Table |
Size | Description
--------+-----------------------------+-------+--------+-----------------+---------+-------------
public | unique_event_type_action_id | index | raj |cust_table | 1256 GB |
Regards,
Raj
On Aug 3, 2020, at 10:20, Konireddy Rajashekar <rajkonireddy@gmail.com> wrote:
Could you please suggest any ideal approach to tackle this ?
You can do CREATE INDEX CONCURRENTLY to build a new index with the same definition, and when that is complete, drop the old index. The locking that is required here is modest: CREATE INDEX CONCURRENTLY needs to lock the table briefly at a couple of points in the operation, and dropping the old index requires a brief lock on the table. It is, however, much less overall lock time than REINDEX would be.
--
-- Christophe Pettus
xof@thebuild.com
On 8/3/20 12:58 PM, Christophe Pettus wrote
On Aug 3, 2020, at 10:20, Konireddy Rajashekar <rajkonireddy@gmail.com> wrote:
Could you please suggest any ideal approach to tackle this ?You can do CREATE INDEX CONCURRENTLY to build a new index with the same definition, and when that is complete, drop the old index. The locking that is required here is modest: CREATE INDEX CONCURRENTLY needs to lock the table briefly at a couple of points in the operation, and dropping the old index requires a brief lock on the table. It is, however, much less overall lock time than REINDEX would be.
Of course, you need enough disk space... :)
--
Angular momentum makes the world go 'round.
creating another index concurrently is taking lot of time
Could you increase maintenance_work_mem significantly or is that already
quite high?
On Mon, Aug 03, 2020 at 01:04:45PM -0500, Ron wrote:
same definition, and when that is complete, drop the old index. The
locking that is required here is modest: CREATE INDEX CONCURRENTLY
needs to lock the table briefly at a couple of points in the
operation, and dropping the old index requires a brief lock on the
table. It is, however, much less overall lock time than REINDEX would be.Of course, you need enough disk space... :)
A SHARE UPDATE EXCLUSIVE lock is taken during a CIC, meaning that
writes and reads are allowed on the parent table while the operation
works, but no DDLs are allowed (roughly). The operation takes a
couple of transactions to complete, and there are two wait points
after building and validating the new index to make sure that there
are no transactions remaining around that may cause visiblity issues
once the new index is ready to use and becomes valid. So the
operation is longer, takes more resources, but it has the advantage to
be non-disruptive.
--
Michael