Index Only Scan vs Cache

Started by Andy Colsonalmost 11 years ago8 messagesgeneral
Jump to latest
#1Andy Colson
andy@squeakycode.net

Hi All.

I have a website db that is 90% read-only. I have 50 (or so) tiny
lookup tables, something like:

\d m_zone
Column | Type | Modifiers
---------+---------+-----------
code | integer | not null
zone_id | text |
descr | text |

This one has less than 10 rows, others might hit 100, I doubt any would
make it over 500 rows.

All of them have an index on code. I'm thinking of dropping it and
creating the index on (code, descr) so that I'd get Index Only Scans.

I host 100 counties (One database, 100 schemas) each will have these 50
tables, so 5,000 small lookup tables.

My question is: Will PG cache only the index (assuming it can always do
an Index Only Scan), or will it cache the table as well?

There is very little difference speed wise, my purpose is to reduce
cache usage. Right now I assume I have 5000 tables + 5000 indexes in cache.

I'd like to try to cut that down to either 5000 tables, or 5000 indexes
(not both).

Thanks for your time,

-Andy

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andy Colson (#1)
Re: Index Only Scan vs Cache

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can always do
an Index Only Scan), or will it cache the table as well?

The table blocks would fall out of cache if they're never touched.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Andy Colson
andy@squeakycode.net
In reply to: Tom Lane (#2)
Re: Index Only Scan vs Cache

On 7/9/2015 12:41 PM, Tom Lane wrote:

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can always do
an Index Only Scan), or will it cache the table as well?

The table blocks would fall out of cache if they're never touched.

regards, tom lane

Sweet! Thanks Tom.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Shujie Shang
sshang@pivotal.io
In reply to: Andy Colson (#3)
Re: Index Only Scan vs Cache

Does PG has its own data cache? I don't think so.
I think PG just using the filesystem cache.

On Fri, Jul 10, 2015 at 2:06 AM, Andy Colson <andy@squeakycode.net> wrote:

Show quoted text

On 7/9/2015 12:41 PM, Tom Lane wrote:

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can always do
an Index Only Scan), or will it cache the table as well?

The table blocks would fall out of cache if they're never touched.

regards, tom lane

Sweet! Thanks Tom.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#5Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shujie Shang (#4)
Re: Index Only Scan vs Cache

2015-07-14 11:59 GMT+02:00 Shujie Shang <sshang@pivotal.io>:

Does PG has its own data cache? I don't think so.

PG has own data cache - see shared_buffers

Regards

Pavel

Show quoted text

I think PG just using the filesystem cache.

On Fri, Jul 10, 2015 at 2:06 AM, Andy Colson <andy@squeakycode.net> wrote:

On 7/9/2015 12:41 PM, Tom Lane wrote:

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can always do
an Index Only Scan), or will it cache the table as well?

The table blocks would fall out of cache if they're never touched.

regards, tom lane

Sweet! Thanks Tom.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Marc Mamin
M.Mamin@intershop.de
In reply to: Andy Colson (#3)
Re: Index Only Scan vs Cache

On 7/9/2015 12:41 PM, Tom Lane wrote:

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can always do
an Index Only Scan), or will it cache the table as well?

I'm not sure that indexes on tiny tables are useful.
They raise the options to consider by the query planner, which has its small cost too.
I'd be interested on other opinions on this.
Any rule of the thumb with which number of pages per relation it is worth to start indexing ?

And still another question: I've have tiny static tables too, that never got analyzed.
Can this fool the query planner in a negative way ?

regards,

Marc Mamin

The table blocks would fall out of cache if they're never touched.

regards, tom lane

Sweet! Thanks Tom.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#7Andy Colson
andy@squeakycode.net
In reply to: Marc Mamin (#6)
Re: Index Only Scan vs Cache

On 7/14/2015 1:19 PM, Marc Mamin wrote:

On 7/9/2015 12:41 PM, Tom Lane wrote:

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can
always do an Index Only Scan), or will it cache the table as
well?

I'm not sure that indexes on tiny tables are useful. They raise the
options to consider by the query planner, which has its small cost
too. I'd be interested on other opinions on this. Any rule of the
thumb with which number of pages per relation it is worth to start
indexing ?

And still another question: I've have tiny static tables too, that
never got analyzed. Can this fool the query planner in a negative way
?

regards,

Marc Mamin

They can be. A unique constraint to ensue correctness for example. In
my case some of my "tiny" tables can be different sizes from 100 to 500
rows. I want indexes on those in case they get bigger and start to get
slow. PG can figure out when to use and not to use the index. I'd
rather have the safety net.

And still another question: I've have tiny static tables too, that never got analyzed.
Can this fool the query planner in a negative way ?

I would say yes. A tiny table is quickest when it is table scanned, but
its only going to be a few milliseconds more if it uses the index (also
depending on how much the table and index are cached). For a small
table I can't imagine the speed difference would even be noticeable.

In my testing, with tables of 100 rows the speed was almost the same
with an index, a covering index, and no index.

-Andy

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#8William Dunn
dunnwjr@gmail.com
In reply to: Marc Mamin (#6)
Re: Index Only Scan vs Cache

On Tue, Jul 14, 2015 at 2:19 PM, Marc Mamin <M.Mamin@intershop.de> wrote:

Any rule of the thumb with which number of pages per relation it is worth
to start indexing ?

The code for the monitoring tool check_postgres uses table size larger than
5.12kb as a rule of thumb, expecting that for tables smaller than 5.12kb
the planner may choose a sequential scan instead because the table is so
small. check_postgres was written a long time ago though, so someone who is
more familiar with the optimizer may be able to provide a better estimate.

*Will J. Dunn*
*willjdunn.com <http://willjdunn.com&gt;*

On Tue, Jul 14, 2015 at 2:19 PM, Marc Mamin <M.Mamin@intershop.de> wrote:

Show quoted text

On 7/9/2015 12:41 PM, Tom Lane wrote:

Andy Colson <andy@squeakycode.net> writes:

My question is: Will PG cache only the index (assuming it can always

do

an Index Only Scan), or will it cache the table as well?

I'm not sure that indexes on tiny tables are useful.
They raise the options to consider by the query planner, which has its
small cost too.
I'd be interested on other opinions on this.
Any rule of the thumb with which number of pages per relation it is worth
to start indexing ?

And still another question: I've have tiny static tables too, that never
got analyzed.
Can this fool the query planner in a negative way ?

regards,

Marc Mamin

The table blocks would fall out of cache if they're never touched.

regards, tom lane

Sweet! Thanks Tom.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general