Why we are going to have to go DirectIO

On Tue, Dec 3, 2013 at 1:44 PM, Josh Berkus <josh@agliodbs.com> wrote:

All,

https://lkml.org/lkml/2013/11/24/133

What this means for us:

http://citusdata.com/blog/72-linux-memory-manager-and-your-big-data

It seems clear that Kernel.org, since 2.6, has been in the business of
pushing major, hackish, changes to the IO stack without testing them or
even thinking too hard about what the side-effects might be. This is
perhaps unsurprising given that two of the largest sponsors of the
Kernel -- who, incidentally, do 100% of the performance testing -- don't
use the IO stack.

This says to me that Linux will clearly be an undependable platform in
the future with the potential to destroy PostgreSQL performance without
warning, leaving us scrambling for workarounds. Too bad the
alternatives are so unpopular.

I don't know where we'll get the resources to implement our own storage,
but it's looking like we don't have a choice.

This seems like a strange reaction to an article that's mostly about
how Linux is now *fixing* a problem that could cause PostgreSQL to
experience performance problems. I agree that we'll probably
eventually need to implement our own storage layer, but this article
isn't evidence of urgency so far as I can determine on first
read-through.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 10:44 AM, Josh Berkus wrote:

All,

https://lkml.org/lkml/2013/11/24/133

What this means for us:

http://citusdata.com/blog/72-linux-memory-manager-and-your-big-data

It seems clear that Kernel.org, since 2.6, has been in the business of
pushing major, hackish, changes to the IO stack without testing them or
even thinking too hard about what the side-effects might be. This is
perhaps unsurprising given that two of the largest sponsors of the
Kernel -- who, incidentally, do 100% of the performance testing -- don't
use the IO stack.

This says to me that Linux will clearly be an undependable platform in
the future with the potential to destroy PostgreSQL performance without
warning, leaving us scrambling for workarounds. Too bad the
alternatives are so unpopular.

I don't know where we'll get the resources to implement our own storage,
but it's looking like we don't have a choice.

This seems rather half cocked. I read the article. They found a problem,
that really will only affect a reasonably small percentage of users,
created a test case, reported it, and a patch was produced.

Kind of like how we do it.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
For my dreams of your image that blossoms
a rose in the deeps of my heart. - W.B. Yeats

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 10:59 AM, Joshua D. Drake wrote:

This seems rather half cocked. I read the article. They found a problem,
that really will only affect a reasonably small percentage of users,
created a test case, reported it, and a patch was produced.

"Users with at least one file bigger than 50% of RAM" is unlikely to be
a small percentage.

Kind of like how we do it.

I like to think we'd have at least researched the existing literature on
2Q algorithms (which is extensive) before implementing our own. Oh,
wait, we *did*. Nor is this the first ill-considered performance hack
pushed into mainline kernels without any real testing. It's not even
the first *that year*.

While I am angry over this -- no matter what Kernel.org fixes now, we're
going to have to live with their mistake for 3 years -- the DirectIO
thing isn't just me; when I've gone to Linux Kernel events to talk about
IO, that's the response I've gotten from most Linux hackers: "you
shouldn't be using the filesystem, use DirectIO and implement your own
storage."

That's why they don't feel that it's a problem to break the IO stack;
they really don't believe that anyone who cares about performance should
be using it.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 08:23 PM, Josh Berkus wrote:

On 12/03/2013 10:59 AM, Joshua D. Drake wrote:

This seems rather half cocked. I read the article. They found a problem,
that really will only affect a reasonably small percentage of users,
created a test case, reported it, and a patch was produced.

"Users with at least one file bigger than 50% of RAM" is unlikely to be
a small percentage.

Kind of like how we do it.

I like to think we'd have at least researched the existing literature on
2Q algorithms (which is extensive) before implementing our own. Oh,
wait, we *did*. Nor is this the first ill-considered performance hack
pushed into mainline kernels without any real testing. It's not even
the first *that year*.

While I am angry over this -- no matter what Kernel.org fixes now, we're
going to have to live with their mistake for 3 years -- the DirectIO
thing isn't just me; when I've gone to Linux Kernel events to talk about
IO, that's the response I've gotten from most Linux hackers: "you
shouldn't be using the filesystem, use DirectIO and implement your own
storage."

That's why they don't feel that it's a problem to break the IO stack;
they really don't believe that anyone who cares about performance should
be using it.

reading that article I think this is an overreaction, it is not
kernel.orgs fault that distributions exist and bugs and regression
happen in all pieces of software.

We are in no way different and I would like to note that we do not have
any form of sensible performance related regression testing either.
I would even argue that there is ton more regression testing (be it
performance or otherwise) going into the linux kernel (even on a
relative scale) than we do and pointing the finger at something they are
dealing with once noticed.
If we care about our performance on various operating systems it is
_OUR_ responsibility to track that closely and automated and report back
and only if that feedback loop fails to work we are actually in a real
position to consider something as drastical as considering a platform
"undependable" or looking into alternatives (like directIO).

Stefan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:

If we care about our performance on various operating systems it is
_OUR_ responsibility to track that closely and automated and report back
and only if that feedback loop fails to work we are actually in a real
position to consider something as drastical as considering a platform
"undependable" or looking into alternatives (like directIO).

+1. I fail to understand why anyone would think it's a good idea for us
to build our own I/O stack. The resources that would be consumed by that
would probably be enough to sink the project, or at least ensure that we
made no progress on any other aspect of the system for a good long time.
(And I'm just talking development, never mind maintenance.)

Far better to invest some effort in providing decent feedback to the
platforms we depend on.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 12:35 PM, Tom Lane wrote:

Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:

If we care about our performance on various operating systems it is
_OUR_ responsibility to track that closely and automated and report back
and only if that feedback loop fails to work we are actually in a real
position to consider something as drastical as considering a platform
"undependable" or looking into alternatives (like directIO).

+1. I fail to understand why anyone would think it's a good idea for us
to build our own I/O stack. The resources that would be consumed by that
would probably be enough to sink the project, or at least ensure that we
made no progress on any other aspect of the system for a good long time.
(And I'm just talking development, never mind maintenance.)

Far better to invest some effort in providing decent feedback to the
platforms we depend on.

Although I am on the same page as Tom and Stefan here, I can certainly
understand Josh's frustration. When you see things like the ext4 bugs or
the recent long slew of performance related issues in relation to
PDFlush, it is enough to make consultants very frustrated with the likes
of Ubuntu and Debian. I would say RedHat too except they learned their
lesson back in the kernel 2.4 days.

Sincerely,

Joshua D. Drake

regards, tom lane

--
Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
For my dreams of your image that blossoms
a rose in the deeps of my heart. - W.B. Yeats

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 12:15 PM, Stefan Kaltenbrunner wrote:

We are in no way different and I would like to note that we do not have
any form of sensible performance related regression testing either.
I would even argue that there is ton more regression testing (be it
performance or otherwise) going into the linux kernel (even on a
relative scale) than we do and pointing the finger at something they are
dealing with once noticed.
If we care about our performance on various operating systems it is
_OUR_ responsibility to track that closely and automated and report back
and only if that feedback loop fails to work we are actually in a real
position to consider something as drastical as considering a platform
"undependable" or looking into alternatives (like directIO).

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com> wrote:

On 12/03/2013 12:15 PM, Stefan Kaltenbrunner wrote:

We are in no way different and I would like to note that we do not have
any form of sensible performance related regression testing either.
I would even argue that there is ton more regression testing (be it
performance or otherwise) going into the linux kernel (even on a
relative scale) than we do and pointing the finger at something they are
dealing with once noticed.
If we care about our performance on various operating systems it is
_OUR_ responsibility to track that closely and automated and report back
and only if that feedback loop fails to work we are actually in a real
position to consider something as drastical as considering a platform
"undependable" or looking into alternatives (like directIO).

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

So in order to get *testing* we need to pay somebody. But to build a great
database server, we can rely on volunteer efforts or sponsorship from
companies who are interested in moving the project forward? That hardly
seems right... Either it's just not high enough on peoples priority lists
(in which case you're not likely to get anybody to actually pay for it
either), or there is some other reason why people just don't care. Figuring
that out would probably be a pre-requisite to get it done. But sure - I'm
all for trying a kickstarter. Did anybody ever try that for an actual
postgres feature? Didn't JD and/or cmd and/or pgus at some point try
something like that?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus@hagander.net> writes:

On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com> wrote:

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

So in order to get *testing* we need to pay somebody. But to build a great
database server, we can rely on volunteer efforts or sponsorship from
companies who are interested in moving the project forward?

And even more to the point, volunteers to reinvent the kernel I/O stack
can be found on every street corner? And those volunteers won't need any
test scaffolding to be sure that *their* version never has performance
regressions? (Well, no, they won't, because no such thing will ever be
built. But we do need better test scaffolding for real problems.)

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Magnus Hagander wrote:

On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com> wrote:

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

So in order to get *testing* we need to pay somebody. But to build a great
database server, we can rely on volunteer efforts or sponsorship from
companies who are interested in moving the project forward?

The reason for this is obvious. You cannot just give the responsibility
of creating a good testing framework to any random guy you just found on
the internets. It has to be an expert, you see.

But sure - I'm all for trying a kickstarter. Did anybody ever try that
for an actual postgres feature? Didn't JD and/or cmd and/or pgus at
some point try something like that?

Hmm, I vaguely recall at CMD there was an attempt to work on a feature
paid via crowd-funding, but I don't recall if we got there or not. In a
way, the foreign key locking patch was done that way, but it wasn't true
crowd-funding but multiple companies sponsoring (at least partly,
because we spent so much more on it than we initially thought we would).

Not quite the same thing, I guess; but then if you get several companies
to each put larger amounts of money than an individual would, I guess
your kickstarter might also succeed.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Magnus,

So in order to get *testing* we need to pay somebody. But to build a great
database server, we can rely on volunteer efforts or sponsorship from
companies who are interested in moving the project forward? That hardly
seems right... Either it's just not high enough on peoples priority lists
(in which case you're not likely to get anybody to actually pay for it
either), or there is some other reason why people just don't care.

It's *always* much easier to get money for features than for other
things. Earlier this year I was really hoping that our new corporate
community members, who seemed to be interested in testing, would put
some serious resources behind this. When pressed, however, they did
what everyone does -- pass and hope that someone else will pay for it.
Huawei staff at least did add a bunch of regression tests, which was
great, but it's a fraction of the work we need for a more comprehensive
testing infrastructure. I got this pretty quickly when Andrew and I led
the session at the unconference. Everybody wanted better testing, but
they all wanted someone else to foot the bill.

We also have the issue that many folks on this list think that testing
isn't important, which further discourages anyone from committing their
own time. But even if the enthusiasm for testing was universal, I think
that we'd need to find money for someone.

I don't think this is prohibitive, though; we do very little fundraising
in this community, and if a testing project had official community
endorsement, I think it would be relatively easy to raise money for it.
Provided that we avoid bikeshedding it to death, of course.

Figuring
that out would probably be a pre-requisite to get it done. But sure - I'm
all for trying a kickstarter. Did anybody ever try that for an actual
postgres feature? Didn't JD and/or cmd and/or pgus at some point try
something like that?

CMD raised money for the FK locks feature. 2Q has raised money for
several features. So has PGX.

I'd rather do the testing thing as a community thing, though, which
means raising non-profit money and having an open bid process for the
person to do the work. I think we cold raise more money that way, and
are likely to get a better end result.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 03:02 PM, Magnus Hagander wrote:

On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

So in order to get *testing* we need to pay somebody. But to build a
great database server, we can rely on volunteer efforts or sponsorship
from companies who are interested in moving the project forward? That
hardly seems right... Either it's just not high enough on peoples
priority lists (in which case you're not likely to get anybody to
actually pay for it either), or there is some other reason why people
just don't care. Figuring that out would probably be a pre-requisite to
get it done. But sure - I'm all for trying a kickstarter. Did anybody
ever try that for an actual postgres feature? Didn't JD and/or cmd
and/or pgus at some point try something like that?

We had our own launched thing that was called FOSSExperts. It wasn't
that successful but it is what started the foreign keys project. That
said, something like KickStarter (or one of the other ones) would have a
lot more exposure.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
For my dreams of your image that blossoms
a rose in the deeps of my heart. - W.B. Yeats

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/03/2013 03:15 PM, Josh Berkus wrote:

It's *always* much easier to get money for features than for other
things. Earlier this year I was really hoping that our new corporate
community members, who seemed to be interested in testing, would put
some serious resources behind this. When pressed, however, they did
what everyone does -- pass and hope that someone else will pay for it.
Huawei staff at least did add a bunch of regression tests, which was
great, but it's a fraction of the work we need for a more comprehensive
testing infrastructure. I got this pretty quickly when Andrew and I led
the session at the unconference. Everybody wanted better testing, but
they all wanted someone else to foot the bill.

+1

I have talked to many a customer and community member about various
things like testing and the overwhelming response is: **crickets** .

It isn't hard to get someone to pay for something that has very little
tangible return, such as a conference sponsorship, but to pay for
something that helps the entire community? **crickets**

There are exceptions of course, but by and large that is my experience.

Sincerely,

Joshua D. Drake

--
Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
For my dreams of your image that blossoms
a rose in the deeps of my heart. - W.B. Yeats

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Magnus Hagander <magnus@hagander.net> writes:

On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com> wrote:

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

So in order to get *testing* we need to pay somebody. But to build a great
database server, we can rely on volunteer efforts or sponsorship from
companies who are interested in moving the project forward?

And even more to the point, volunteers to reinvent the kernel I/O stack
can be found on every street corner? And those volunteers won't need any
test scaffolding to be sure that *their* version never has performance
regressions? (Well, no, they won't, because no such thing will ever be
built. But we do need better test scaffolding for real problems.)

Can we avoid the Linux kernel problem by simply increasing our shared
buffer size, say up to 80% of memory?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

(2013/12/04 11:28), Tatsuo Ishii wrote:

Magnus Hagander <magnus@hagander.net> writes:

On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com> wrote:

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has never
been available; maybe I should do a kickstarter.

So in order to get *testing* we need to pay somebody. But to build a great
database server, we can rely on volunteer efforts or sponsorship from
companies who are interested in moving the project forward?

And even more to the point, volunteers to reinvent the kernel I/O stack
can be found on every street corner? And those volunteers won't need any
test scaffolding to be sure that *their* version never has performance
regressions? (Well, no, they won't, because no such thing will ever be
built. But we do need better test scaffolding for real problems.)

Can we avoid the Linux kernel problem by simply increasing our shared
buffer size, say up to 80% of memory?

It will be swap more easier.

I think that we should use latest system-calls in Linux which are like
posix_fadvise(), fallocate() and sync_file_range() etc, when we use linux
buffered IO. Hoevere, PostgreSQL doesn't use these system-call a lots.
Especially, I think that checkpoint algorithm is very ugly..

Regards,
--
Mitsumasa KONDO
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, 2013-12-03 at 14:44 -0800, Josh Berkus wrote:

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has
never been available; maybe I should do a kickstarter.

I think the problem is, it's not even clear what the deliverable might
be. Benchmarking tools exist, and running them on a regular schedule
shouldn't be difficult. But that doesn't find regressions between
kernel versions, for example, or regressions in particular queries
(unless they happen to be included in the benchmark).

The first step here should be to work out the minimum viable product,
and then see what it would take to get that done.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/04/2013 05:40 AM, Peter Eisentraut wrote:

On Tue, 2013-12-03 at 14:44 -0800, Josh Berkus wrote:

Would certainly be nice. Realistically, getting good automated
performace tests will require paying someone like Greg S., Mark or me
for 6 solid months to develop them, since worthwhile open source
performance test platforms currently don't exist. That money has
never been available; maybe I should do a kickstarter.

I think the problem is, it's not even clear what the deliverable might
be. Benchmarking tools exist, and running them on a regular schedule
shouldn't be difficult. But that doesn't find regressions between
kernel versions, for example, or regressions in particular queries
(unless they happen to be included in the benchmark).

I agree on the problem of specifying an exact deliverable - however
simple using some of the extisting benchmark tool and maybe augment them
by the myriad of simple "micro" level regressions we have in the form of
sql queries in the archives would be a sensible start. It might not help
for all cases but it can help for some and we learn something that might
help us building the next iteration of it. Adding say some operatimng
systems to the mix of we have the above would be fairly easy - running a
few kvm instances that get bootstrapped automatically is something that
is a solved problem.

The first step here should be to work out the minimum viable product,
and then see what it would take to get that done.

yeah we need to start somewhere and see what we can learn.

Stefan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Can we avoid the Linux kernel problem by simply increasing our shared
buffer size, say up to 80% of memory?

It will be swap more easier.

Is that the case? If the system has not enough memory, the kernel
buffer will be used for other purpose, and the kernel cache will not
work very well anyway. In my understanding, the problem is, even if
there's enough memory, the kernel's cache does not work as expected.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 4, 2013 at 4:28 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

Can we avoid the Linux kernel problem by simply increasing our shared
buffer size, say up to 80% of memory?

It will be swap more easier.

Is that the case? If the system has not enough memory, the kernel
buffer will be used for other purpose, and the kernel cache will not
work very well anyway. In my understanding, the problem is, even if
there's enough memory, the kernel's cache does not work as expected.

Problem is, Postgres relies on a working kernel cache for checkpoints.
Checkpoint logic would have to be heavily reworked to account for an
impaired kernel cache.

Really, there's no difference between fixing the I/O problems in the
kernel(s) vs in postgres. The only difference is, in the kernel(s),
everyone profits, and you've got a huge head start.

Communicating more with the kernel (through posix_fadvise, fallocate,
aio, iovec, etc...) would probably be good, but it does expose more
kernel issues. posix_fadvise, for instance, is a double-edged sword
ATM. I do believe, however, that exposing those issues and prompting a
fix is far preferable than silently working around them.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers