Parallel GiST build on Cube
Hello --
I regularly build GiST indexes on large databases. In recent times, as the
size of the database has ballooned (30 million rows) along with the build
time on a column of points in 3- and 8-dimensional space (0-volume cube).
Is anyone working on (or already implemented) a parallel GiST build on
Cube? If not, I'd like to try contributing this--any pointers from folks
familiar with the backend of Cube? Any input would be great.
Thanks,
Shyam
--
Shyam Saladi <http://shyam.saladi.org>
NSF Graduate Research Fellow - Clemons Lab
Biochemistry and Molecular Biophysics
California Institute of Technology
Hello,
These things for GIST I know that can help:
- Fast sorting GIST build commitfest entry by Andrey Borodin, not parallel
but faster -
/messages/by-id/1A36620E-CAD8-4267-9067-FB31385E7C0D@yandex-team.ru
- Fast sorting GIST build by Nikita Glukhov, reuses btree code so also
parallel -
https://github.com/postgres/postgres/compare/master...glukhovn:gist_btree_build
- "Choose Subtree" routine is needed, as current "penalty" is very
inefficient -
/messages/by-id/CAPpHfdssv2i7CXTBfiyR6=_A3tp19+iLo-pkkfD6Guzs2-tvEw@mail.gmail.com
These are very wanted for PostGIS which also indexes everything by 2-4
dimensional cubes and require improvements in core infrastructure and
opclass.
On Mon, Apr 27, 2020 at 8:57 PM Shyam Saladi <saladi@caltech.edu> wrote:
Hello --
I regularly build GiST indexes on large databases. In recent times, as the
size of the database has ballooned (30 million rows) along with the build
time on a column of points in 3- and 8-dimensional space (0-volume cube).Is anyone working on (or already implemented) a parallel GiST build on
Cube? If not, I'd like to try contributing this--any pointers from folks
familiar with the backend of Cube? Any input would be great.Thanks,
Shyam--
Shyam Saladi <http://shyam.saladi.org>
NSF Graduate Research Fellow - Clemons Lab
Biochemistry and Molecular Biophysics
California Institute of Technology
--
Darafei Praliaskouski
Support me: http://patreon.com/komzpa