Unlogged vs. In-Memory

On Tue, May 3, 2011 at 6:46 PM, Joshua Berkus <josh@agliodbs.com> wrote:

Saying "It's like a in-memory table" is a lot more successful.  And it's using the term "in-memory" the same way a lot of other DBMSes market it, i.e. in-memory == non-durable & no disk writes.

Unlogged tables *are* written to disk though. And may be read from
there too - they are not pinned into memory.

The important thing from my perspective is that unlogged tables give us the capabilities of a lot of the "in-memory" databases ... with unlogged tables and fsync off, for example, PostgreSQL becomes a viable caching database.

When doing PR, it's more important to use terms people recognize than to use terms which are perfectly accurate.  Nobody expects a news article to be perfectly accurate anyway.

I consider the term to be pretty much 100% inaccurate. When you say
in-memory to me, I think of a table that is pinned into buffer cache,
as you can do in some DBMS', thus *ensuring* the reads are fast, or of
a database or table that operates entirely in memory (perhaps with
occasional disk writes for persistence) like VoltDB, Redis or MySQL's
Memory storage manager.

However, I posted this because I think that several folks in the community feel that this is going too far into the land of marketese, and I want to hash it out and get consensus before we start pitching 9.1 final.

Thank you.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 3 May 2011 18:46, Joshua Berkus <josh@agliodbs.com> wrote:

All,

This has come up a couple times off-list, so I thought we should hammer it out here regarding messaging for 9.1.

I was discussing the Unlogged Tables feature with an industry analyst.  He advised me fairly strongly that we should call it, or at least describe it, as "in-memory tables".  While I'm not that sanguine about renaming the feature, I'm happy to use marketing terms in descriptive text in a press release if it gets people interested.

Our basic issue with the cool features in 9.1 is the elevator pitch problem.  Try to describe SSI to a reporter in 20 seconds or less.  Unlogged tables suffers from this.  "What's an unlogged table? Why is *not* having something a feature?"  "long description here ..." "nevermind, I have enough."

Saying "It's like a in-memory table" is a lot more successful.  And it's using the term "in-memory" the same way a lot of other DBMSes market it, i.e. in-memory == non-durable & no disk writes.  The important thing from my perspective is that unlogged tables give us the capabilities of a lot of the "in-memory" databases ... with unlogged tables and fsync off, for example, PostgreSQL becomes a viable caching database.

When doing PR, it's more important to use terms people recognize than to use terms which are perfectly accurate.  Nobody expects a news article to be perfectly accurate anyway.

However, I posted this because I think that several folks in the community feel that this is going too far into the land of marketese, and I want to hash it out and get consensus before we start pitching 9.1 final.

As far as I'm aware, an unlogged table is just a table which
sacrifices crash safety for speed. It's not "in-memory" because that
suggests it's always kept in the physical memory, which isn't the
case.

--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Joshua Berkus <josh@agliodbs.com> wrote:

I was discussing the Unlogged Tables feature with an industry
analyst. He advised me fairly strongly that we should call it, or
at least describe it, as "in-memory tables". While I'm not that
sanguine about renaming the feature, I'm happy to use marketing
terms in descriptive text in a press release if it gets people
interested.

I guess it does get the main idea across at the front. We could
include in the "fine print" that the in-memory data can be paged to
disk temporarily when memory is needed for other purposes, and that
it will be saved on clean shutdown for automatic reload when
possible.

Our basic issue with the cool features in 9.1 is the elevator
pitch problem. Try to describe SSI to a reporter in 20 seconds or
less.

Yeah, I've noticed that ...

Nobody expects a news article to be perfectly accurate anyway.

... and that.

However, I posted this because I think that several folks in the
community feel that this is going too far into the land of
marketese, and I want to hash it out and get consensus before we
start pitching 9.1 final.

As long as the end result is accurate if someone makes it through
the whole thing, I don't think it's a problem to lead with the main
point. In other words, calling it an in-memory table does capture
the essence of the intent; it is enough if the caveats which come
later cover the exceptions, IMO. But let's not rename the feature;
this is about marketing presentation.

-Kevin

Can Unlogged tables be located on a table space mount on a ram fs
without hosing the instance if the server gets bounced?

On Tue, May 3, 2011 at 10:46 AM, Joshua Berkus <josh@agliodbs.com> wrote:

All,

This has come up a couple times off-list, so I thought we should hammer it out here regarding messaging for 9.1.

I was discussing the Unlogged Tables feature with an industry analyst.  He advised me fairly strongly that we should call it, or at least describe it, as "in-memory tables".  While I'm not that sanguine about renaming the feature, I'm happy to use marketing terms in descriptive text in a press release if it gets people interested.

Our basic issue with the cool features in 9.1 is the elevator pitch problem.  Try to describe SSI to a reporter in 20 seconds or less.  Unlogged tables suffers from this.  "What's an unlogged table? Why is *not* having something a feature?"  "long description here ..." "nevermind, I have enough."

Saying "It's like a in-memory table" is a lot more successful.  And it's using the term "in-memory" the same way a lot of other DBMSes market it, i.e. in-memory == non-durable & no disk writes.  The important thing from my perspective is that unlogged tables give us the capabilities of a lot of the "in-memory" databases ... with unlogged tables and fsync off, for example, PostgreSQL becomes a viable caching database.

When doing PR, it's more important to use terms people recognize than to use terms which are perfectly accurate.  Nobody expects a news article to be perfectly accurate anyway.

However, I posted this because I think that several folks in the community feel that this is going too far into the land of marketese, and I want to hash it out and get consensus before we start pitching 9.1 final.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
San Francisco

--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

--
Rob Wultsch
wultsch@gmail.com

On 3 May 2011 19:02, Rob Wultsch <wultsch@gmail.com> wrote:

Can Unlogged tables be located on a table space mount on a ram fs
without hosing the instance if the server gets bounced?

No more than anything else in a RAM filesystem. There are of course
battery-backed RAM disk devices people can use, but those are a
special case.

--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 5/3/11 11:01 AM, Kevin Grittner wrote:

In other words, calling it an in-memory table does capture
the essence of the intent; it is enough if the caveats which come
later cover the exceptions, IMO. But let's not rename the feature;
this is about marketing presentation.

Right. What I'm suggesting ... and have already been doing, because I
didn't realize it would be a problem, is that we say something like this
in the description:

"Unlogged tables are similar to in-memory tables or global temporary
tables."

That way, we make it clear that they're not exactly the same, but we
still use the right buzzwords. And they are similar, because they can
be used to fill the same needs.

Part of the problem is the name we're using for the feature. "Unlogged
tables" sounds like we've taken something away and are calling that a
feature. "Now with no brakes!" As feature names go, it's as unsexy as
you can get.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Tue, May 3, 2011 at 6:46 PM, Joshua Berkus <josh@agliodbs.com> wrote:

This has come up a couple times off-list, so I thought we should hammer it out here regarding messaging for 9.1.

I was discussing the Unlogged Tables feature with an industry analyst.  He advised me fairly strongly that we should call it, or at least describe it, as "in-memory tables".  While I'm not that sanguine about renaming the feature, I'm happy to use marketing terms in descriptive text in a press release if it gets people interested.

Our basic issue with the cool features in 9.1 is the elevator pitch problem.  Try to describe SSI to a reporter in 20 seconds or less.  Unlogged tables suffers from this.  "What's an unlogged table? Why is *not* having something a feature?"  "long description here ..." "nevermind, I have enough."

Saying "It's like a in-memory table" is a lot more successful.  And it's using the term "in-memory" the same way a lot of other DBMSes market it, i.e. in-memory == non-durable & no disk writes.  The important thing from my perspective is that unlogged tables give us the capabilities of a lot of the "in-memory" databases ... with unlogged tables and fsync off, for example, PostgreSQL becomes a viable caching database.

When doing PR, it's more important to use terms people recognize than to use terms which are perfectly accurate.  Nobody expects a news article to be perfectly accurate anyway.

However, I posted this because I think that several folks in the community feel that this is going too far into the land of marketese, and I want to hash it out and get consensus before we start pitching 9.1 final.

The only reason not to do this is that they aren't actually "In
Memory". Unlogged tables can be any size and therefore scroll to disk.

Now admittedly, a very large UNLOGGED table is about as robust as a
chocolate teapot, so I do see where you're coming from.

I'm at a loss to understand why the implementation is so different
from other DBMS
http://www.dba-oracle.com/oracle_news/2005_6_10_NOLOGGING_NOARCHIVELOG_Standby_Databases_Part%201.htm

With regards to a renaming, maybe we should call them CACHE TABLES, to
indicate what they are useful for. At least that doesn't confuse
things more by implying they are just in-memory.

The problem I see is that there isn't any event that fires when the
system crashes and zeroes all the tables, so if you do use it for a
cache, you must always use check-cache-or-read-from-main-source logic
because the cache isn't refilled automatically. In fact there is no
way to tell it has crashed at all, AFAICS, except by having a 1 row
table that would be empty in case of a crash.

I agree that we probably need to rename or rework them as they stand.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On Tue, May 3, 2011 at 7:08 PM, simon <simon@2ndQuadrant.com> wrote:

I agree that we probably need to rename or rework them as they stand.

...and, as you know, there are minor changes I'd like to make before
we release to my own work to round out the features we are delivering,
so these comments aren't partisan.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

El mar, 03-05-2011 a las 12:46 -0500, Joshua Berkus escribió:

All,

This has come up a couple times off-list, so I thought we should hammer it out here regarding messaging for 9.1.

"in-memory tables" song good.

--
Saludos,
Gilberto Castillo
Edificio Beijing. Miramar Trade Center. Etecsa.
Miramar, La Habana.Cuba.

Part of the problem is the name we're using for the feature. "Unlogged
tables" sounds like we've taken something away and are calling that a
feature. "Now with no brakes!" As feature names go, it's as unsexy as

Logless tables?

Log-Free tables?

On Tue, May 3, 2011 at 20:06, Thom Brown <thom@linux.com> wrote:

On 3 May 2011 19:02, Rob Wultsch <wultsch@gmail.com> wrote:

Can Unlogged tables be located on a table space mount on a ram fs
without hosing the instance if the server gets bounced?

No more than anything else in a RAM filesystem.  There are of course
battery-backed RAM disk devices people can use, but those are a
special case.

I think you're missing the scenario Rob is talking about. I think he
mentions the sequence:

CREATE TABLESPACE junk LOCATION '/tmp/junk';
CREATE UNLOGGED TABLE meh(a int) TABLESPACE junk;
<stop server>
rm -rf /tmp/junk/*
<start server>
postgres=# select * from meh;
ERROR: could not open file
"pg_tblspc/16434/PG_9.1_201104251/12008/16435": No such file or
directory

Now if the tablespace contains *only* unlogged tables, it should at
least theoretically be possible to recover from this situation on
startup, I think. But it's not now. Anybody have an idea about how
much work that would be?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Tue, May 3, 2011 at 20:07, Josh Berkus <josh@agliodbs.com> wrote:

On 5/3/11 11:01 AM, Kevin Grittner wrote:

In other words, calling it an in-memory table does capture
the essence of the intent; it is enough if the caveats which come
later cover the exceptions, IMO.  But let's not rename the feature;
this is about marketing presentation.

Right.   What I'm suggesting ... and have already been doing, because I
didn't realize it would be a problem, is that we say something like this
in the description:

"Unlogged tables are similar to in-memory tables or global temporary
tables."

They are *not* similar to in-memory table, in that they are *always*
written to disk. AFAIK that is - or do they actually get spooled in
RAM-only until they get big enough? I'm prettysure they don't.

They *are*, however, pretty similar to global temporary tables. Are
those well known enough to be used for the pitch without mentioning
in-memory tables?

Part of the problem is the name we're using for the feature.  "Unlogged
tables" sounds like we've taken something away and are calling that a
feature.  "Now with no brakes!"  As feature names go, it's as unsexy as
you can get.

"nosql tables"? ;)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Tue, May 3, 2011 at 7:45 PM, Magnus Hagander <magnus@hagander.net> wrote:

On Tue, May 3, 2011 at 20:07, Josh Berkus <josh@agliodbs.com> wrote:

On 5/3/11 11:01 AM, Kevin Grittner wrote:

In other words, calling it an in-memory table does capture
the essence of the intent; it is enough if the caveats which come
later cover the exceptions, IMO.  But let's not rename the feature;
this is about marketing presentation.

Right.   What I'm suggesting ... and have already been doing, because I
didn't realize it would be a problem, is that we say something like this
in the description:

"Unlogged tables are similar to in-memory tables or global temporary
tables."

They are *not* similar to in-memory table, in that they are *always*
written to disk. AFAIK that is - or do they actually get spooled in
RAM-only until they get big enough? I'm prettysure they don't.

They *are*, however, pretty similar to global temporary tables. Are
those well known enough to be used for the pitch without mentioning
in-memory tables?

I wouldn't. Robert and I talked a number of times about him
implementing global temp tables after unlogged tables. If he does,
then things will just get a whole heap more confusing next year if
we're already (mis)used the term.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Magnus Hagander <magnus@hagander.net> wrote:

They are *not* similar to in-memory table, in that they are
*always* written to disk.

I thought we avoided flushing them to disk on checkpoint, or did
that idea fall flat? Does the background writer flush them? If
neither of these happens, then we can legitimately call them
in-memory, as long as we point out that they are saved on a clean
shutdown for reload on startup, and may be flushed from RAM at times
when other objects need the memory.

-Kevin

On 5/3/11 11:45 AM, Magnus Hagander wrote:

They are *not* similar to in-memory table, in that they are *always*
written to disk.

Only if they need to get pushed out of RAM, no?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Tue, May 3, 2011 at 20:52, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

Magnus Hagander <magnus@hagander.net> wrote:

They are *not* similar to in-memory table, in that they are
*always* written to disk.

I thought we avoided flushing them to disk on checkpoint, or did
that idea fall flat?  Does the background writer flush them?  If
neither of these happens, then we can legitimately call them
in-memory, as long as we point out that they are saved on a clean
shutdown for reload on startup, and may be flushed from RAM at times
when other objects need the memory.

I thought that wasn't implemented. But I could certainly have missed
something around it. If they are like that then yes, we can probably
get around calling them "similar to" in-memory tables.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 3 May 2011 19:45, Magnus Hagander <magnus@hagander.net> wrote:

On Tue, May 3, 2011 at 20:07, Josh Berkus <josh@agliodbs.com> wrote:

Part of the problem is the name we're using for the feature.  "Unlogged
tables" sounds like we've taken something away and are calling that a
feature.  "Now with no brakes!"  As feature names go, it's as unsexy as
you can get.

"nosql tables"? ;)

In all seriousness, I think it's important we should not mention
anything like that at any point. :S

--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, May 3, 2011 at 7:45 PM, Magnus Hagander <magnus@hagander.net> wrote:

"Unlogged tables are similar to in-memory tables or global temporary
tables."

They are *not* similar to in-memory table, in that they are *always*
written to disk. AFAIK that is - or do they actually get spooled in
RAM-only until they get big enough? I'm prettysure they don't.

They *are*, however, pretty similar to global temporary tables. Are
those well known enough to be used for the pitch without mentioning
in-memory tables?

Apparently not.

Part of the problem is the name we're using for the feature.  "Unlogged
tables" sounds like we've taken something away and are calling that a
feature.  "Now with no brakes!"  As feature names go, it's as unsexy as
you can get.

"nosql tables"? ;)

Not that either.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On Tue, May 3, 2011 at 2:28 PM, Joshua Kramer <josh@globalherald.net> wrote:

Part of the problem is the name we're using for the feature.  "Unlogged
tables" sounds like we've taken something away and are calling that a
feature.  "Now with no brakes!"  As feature names go, it's as unsexy as

Logless tables?
Log-Free tables?

The best way to show off a new feature is to emphasize the positive
aspects. The main reason people will use unlogged tables is to improve
performance on tables that do not need to be crash safe. I would
propose calling the feature something like "Fast Tables", and the fine
print can mention the trade-offs related to not logging.

Just my thoughts,
~Ian