Regarding GIN Fast Update Technique

On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:

At what point do the entries in the pending list are moved to the main
GIN data structure?
From documentation, I read that overflowing work_mem and vacuum are
two such causes; what about when the concerned backend is to exit and
autovacuum has not yet kicked in?

I don't think there's any special handling for that case, nor do I
think any is needed.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:

At what point do the entries in the pending list are moved to the main
GIN data structure?
From documentation, I read that overflowing work_mem and vacuum are
two such causes; what about when the concerned backend is to exit and
autovacuum has not yet kicked in?

I don't think there's any special handling for that case, nor do I
think any is needed.

Okay, aside from that case, what else would move those to the main
structure? They (the entries in the unsorted pending list) are in the
local memory (work_mem?) of the backend, right?

--
Amit Langote

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-06-07 23:28:56 +0900, Amit Langote wrote:

On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:

At what point do the entries in the pending list are moved to the main
GIN data structure?
From documentation, I read that overflowing work_mem and vacuum are
two such causes; what about when the concerned backend is to exit and
autovacuum has not yet kicked in?

I don't think there's any special handling for that case, nor do I
think any is needed.

Okay, aside from that case, what else would move those to the main
structure? They (the entries in the unsorted pending list) are in the
local memory (work_mem?) of the backend, right?

No, it's in the normal heap, pointed to by the metapage. Storing it in
local memory would mean we would have to flush it out before commit.

That part of gin is actually quite readable code, so I suggest looking
there. Start with ginfast.c:ginHeapTupleFastInsert().

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Amit Langote <amitlangote09@gmail.com> writes:

Okay, aside from that case, what else would move those to the main
structure? They (the entries in the unsorted pending list) are in the
local memory (work_mem?) of the backend, right?

No. If they were, it wouldn't be crash-safe.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Jun 7, 2013 at 11:36 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2013-06-07 23:28:56 +0900, Amit Langote wrote:

On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:

At what point do the entries in the pending list are moved to the main
GIN data structure?
From documentation, I read that overflowing work_mem and vacuum are
two such causes; what about when the concerned backend is to exit and
autovacuum has not yet kicked in?

I don't think there's any special handling for that case, nor do I
think any is needed.

Okay, aside from that case, what else would move those to the main
structure? They (the entries in the unsorted pending list) are in the
local memory (work_mem?) of the backend, right?

No, it's in the normal heap, pointed to by the metapage. Storing it in
local memory would mean we would have to flush it out before commit.

That part of gin is actually quite readable code, so I suggest looking
there. Start with ginfast.c:ginHeapTupleFastInsert().

Thanks.

--
Amit Langote

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 07/06/13 16:39, Tom Lane wrote:

Amit Langote <amitlangote09@gmail.com> writes:

Okay, aside from that case, what else would move those to the main
structure? They (the entries in the unsorted pending list) are in the
local memory (work_mem?) of the backend, right?

No. If they were, it wouldn't be crash-safe.

Thats how it is, but if we could push in wishes, then I would
wish that is woulndn't be crash-safe, and be flushed by the backends
commit. The way it currently operates is that a "random backend"
pays the penalty of other backends pushes to the pending-list and "all
queries"
pays the penalty of searching the pendinglist in queries.

If the pending list were backend only it would not have to be searched
by "all queries" since commit needs to flush it and random backends
wouldn't be penalized. Allthough we'd still have the benefit of batching up
gin-inserts over mulitiple changes to the index done within the same
transaction.

Jesper

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers