Idea: Always consistent in-database cache using SSI mechanisms

Alexander Korotkov <aekorotkov@gmail.com> wrote:

Coundn't be predicate locking implementation in SSI be used for
in-database cache invalidation.

It would not necessarily be limited to *in-database* caches. The
main thing would be to design a good API to the predicate locking
portion of SSI, which I think is about 80% of the SSI code. Dan and
I both have an interest in such further use, and there have been
others who have talked about potential uses for the non-blocking
predicate locking. I think the API would need to be based around a
listen/notify model.

It could be possible to implement in-database cache which will
acquire predicate locks like SSI transactions. In case of any
conflich with other transactions corresponding cache invalidates.
Therefore, it might be possible to get acceleration of caching
without risk of inconsistent answers.

I had not thought of that potential use. At first glance, I think
it has possibilities, but only if the above-mentioned API was
formalized *and* there was some way to configure a cluster for
"serializable transactions only". Long-range, I have hopes for
both.

Actually, I don't understand SSI in details. So, probably I mess
up things. Does my idea any matter?

Sure! Besides having the available development time, I think the
biggest obstacle is having enough plausible use cases for predicate
lock access to do a good job defining the API. While we made some
effort to keep the predicate locking and serializable behavior
separate in the implementation, it wasn't clear where the "bright
line" was, so there is bound to be some rearrangement needed when we
figure that out. The more ideas we have in front of us on how
predicate locks might be useful, the better the API design is likely
to be.

-Kevin

On Tue, Oct 25, 2011 at 1:46 AM, Kevin Grittner <Kevin.Grittner@wicourts.gov

wrote:

Alexander Korotkov <aekorotkov@gmail.com> wrote:

Coundn't be predicate locking implementation in SSI be used for
in-database cache invalidation.

It would not necessarily be limited to *in-database* caches. The
main thing would be to design a good API to the predicate locking
portion of SSI, which I think is about 80% of the SSI code. Dan and
I both have an interest in such further use, and there have been
others who have talked about potential uses for the non-blocking
predicate locking. I think the API would need to be based around a
listen/notify model.

It could be possible to implement in-database cache which will
acquire predicate locks like SSI transactions. In case of any
conflich with other transactions corresponding cache invalidates.
Therefore, it might be possible to get acceleration of caching
without risk of inconsistent answers.

I had not thought of that potential use. At first glance, I think
it has possibilities, but only if the above-mentioned API was
formalized *and* there was some way to configure a cluster for
"serializable transactions only". Long-range, I have hopes for
both.

Sure, it would be rather better to implement that through API.

Actually, I don't understand SSI in details. So, probably I mess

up things. Does my idea any matter?

Sure! Besides having the available development time, I think the
biggest obstacle is having enough plausible use cases for predicate
lock access to do a good job defining the API. While we made some
effort to keep the predicate locking and serializable behavior
separate in the implementation, it wasn't clear where the "bright
line" was, so there is bound to be some rearrangement needed when we
figure that out. The more ideas we have in front of us on how
predicate locks might be useful, the better the API design is likely
to be.

Thanks for feedback on my idea. I'll share ideas about more possible usage
of that API if I have any.

------
With best regards,
Alexander Korotkov.

On Tue, Oct 25, 2011 at 00:00, Alexander Korotkov <aekorotkov@gmail.com> wrote:

On Tue, Oct 25, 2011 at 1:46 AM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

Alexander Korotkov <aekorotkov@gmail.com> wrote:

Coundn't be predicate locking implementation in SSI be used for
in-database cache invalidation.

It would not necessarily be limited to *in-database* caches.  The
main thing would be to design a good API to the predicate locking
portion of SSI, which I think is about 80% of the SSI code.  Dan and
I both have an interest in such further use, and there have been
others who have talked about potential uses for the non-blocking
predicate locking.  I think the API would need to be based around a
listen/notify model.

IIRC, I discussed this with Dan Ports at pgcon, as it was one of the
things he had been looking into as well. You might want to talk to him
about it.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/