O_DIRECT and performance
Well, O_DIRECT has finally made it into the Linux kernel. It lets you
open a file in such a way that reads and writes don't go to the buffer
cache but straight to the disk. Accesses must be aligned on
filesystem block boundaries.
Is there any case where PG would benefit from this? I can see it
reducing memory pressure and duplication of data between PG shared
buffers and the block buffer cache. OTOH, it does require that writes
be batched up for decent performance, since each write has an implicit
fsync() involved (just as with O_SYNC).
Anyone played with this on systems that already support it (Solaris?)
-Doug
--
In a world of steel-eyed death, and men who are fighting to be warm,
Come in, she said, I'll give you shelter from the storm. -Dylan
Well, O_DIRECT has finally made it into the Linux kernel. It lets you
open a file in such a way that reads and writes don't go to the buffer
cache but straight to the disk. Accesses must be aligned on
filesystem block boundaries.Is there any case where PG would benefit from this? I can see it
reducing memory pressure and duplication of data between PG shared
buffers and the block buffer cache. OTOH, it does require that writes
be batched up for decent performance, since each write has an implicit
fsync() involved (just as with O_SYNC).Anyone played with this on systems that already support it (Solaris?)
I have heard there are many cases there O_DIRECT on Solaris is slower
for databases than normal I/O. I think bulk copy was faster but not
normal operation. Probably not something we are going to get into soon.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
Red Hat Linux 7.2, postgresql 7.2b2 configured like this:
./configure --enable-syslog --prefix=/usr --sysconfdir=/etc --with-pam
--with-openssl --with-unixodbc --with-tcl --with-perl --with-python
--enable-locale --enable-recode --enable-multibyte --enable-nls
"make check" fails on initdb:
============== creating temporary installation ==============
============== initializing database system ==============
pg_regress: initdb failed
Examine ./log/initdb.log for the reason.
teg@halden postgresql-7.2b2]$ cat src/test/regress/log/initdb.log
Running with noclean mode on. Mistakes will not be cleaned up.
/home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: undefined symbol: pg_valid_server_encoding
initdb: pg_encoding failed
Perhaps you did not configure PostgreSQL for multibyte support or
the program was not successfully installed.
[teg@halden postgresql-7.2b2]$
--
Trond Eivind Glomsr�d
Red Hat, Inc.
teg@redhat.com (Trond Eivind Glomsr�d) writes:
Red Hat Linux 7.2, postgresql 7.2b2 configured like this:
./configure --enable-syslog --prefix=/usr --sysconfdir=/etc --with-pam
--with-openssl --with-unixodbc --with-tcl --with-perl --with-python
--enable-locale --enable-recode --enable-multibyte --enable-nls
When just using "configure" it works - apart from a false failure in
the regression tests (geometry), it seems to work (a couple cases of
"-0.121320343559642" vs. "-0.121320343559643" - a better way than raw
diff for comparing would be useful).
--
Trond Eivind Glomsr�d
Red Hat, Inc.
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
/home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: undefined symbol: pg_valid_server_encoding
initdb: pg_encoding failed
pg_encoding relies on libpq to supply the pg_valid_server_encoding()
subroutine, but that subroutine is only compiled into libpq in a
MULTIBYTE build. I speculate that your executable was picking up
a non-MULTIBYTE libpq shared library from someplace. Check ldconfig
and all that stuff...
FWIW, both multibyte and non-multibyte builds are working OK for me
in current sources.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes:
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
/home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: undefined symbol: pg_valid_server_encoding
initdb: pg_encoding failedpg_encoding relies on libpq to supply the pg_valid_server_encoding()
subroutine, but that subroutine is only compiled into libpq in a
MULTIBYTE build. I speculate that your executable was picking up
a non-MULTIBYTE libpq shared library from someplace. Check ldconfig
and all that stuff...
I have an existing installation of 7.1 on the system, that's why I did
"make check" in the build directory.
"--prefix=/usr" seems to be the "culprit" - without it, it regression
tests run just fine.
--
Trond Eivind Glomsr�d
Red Hat, Inc.
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
Tom Lane <tgl@sss.pgh.pa.us> writes:
I speculate that your executable was picking up
a non-MULTIBYTE libpq shared library from someplace. Check ldconfig
and all that stuff...
I have an existing installation of 7.1 on the system, that's why I did
"make check" in the build directory.
"--prefix=/usr" seems to be the "culprit" - without it, it regression
tests run just fine.
The pg_regress script sets LD_LIBRARY_PATH to try to cause libpq and
the other shlibs to be picked up from the temp installation tree.
Perhaps this is wrong or insufficient on your platform. It certainly
sounds like the dynamic linker is choosing the installed libpq over
the one that we want it to use. Any thoughts on fixing that?
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes:
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
Tom Lane <tgl@sss.pgh.pa.us> writes:
I speculate that your executable was picking up
a non-MULTIBYTE libpq shared library from someplace. Check ldconfig
and all that stuff...I have an existing installation of 7.1 on the system, that's why I did
"make check" in the build directory."--prefix=/usr" seems to be the "culprit" - without it, it regression
tests run just fine.The pg_regress script sets LD_LIBRARY_PATH to try to cause libpq and
the other shlibs to be picked up from the temp installation tree.
Perhaps this is wrong or insufficient on your platform. It certainly
sounds like the dynamic linker is choosing the installed libpq over
the one that we want it to use. Any thoughts on fixing that?
Since it works when prefix isn't /usr, I'd guess that the build
process sets rpath. This takes precedence over LD_LIBRARY_PATH.
Fix? Don't use rpath - it's evil, and should be avoided anyway.
ld(1) contains some info on how libraries are resolved.
--
Trond Eivind Glomsr�d
Red Hat, Inc.
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
Since it works when prefix isn't /usr, I'd guess that the build
process sets rpath. This takes precedence over LD_LIBRARY_PATH.
Fix? Don't use rpath - it's evil, and should be avoided anyway.
Peter, any thoughts on that? It's not clear to me that removing
rpath settings would be an improvement overall, even if it makes
"make check" more reliable.
regards, tom lane
* Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]:
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
Since it works when prefix isn't /usr, I'd guess that the build
process sets rpath. This takes precedence over LD_LIBRARY_PATH.Fix? Don't use rpath - it's evil, and should be avoided anyway.
Peter, any thoughts on that? It's not clear to me that removing
rpath settings would be an improvement overall, even if it makes
"make check" more reliable.
My general take is rpath is a *GOOD* thing, as it prevents USERS from
not being able to find REQUIRED shared libs.
I vote for it staying.
LER
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Trond Eivind Glomsr�d writes:
Fix? Don't use rpath - it's evil, and should be avoided anyway.
configure --disable-rpath is your friend.
Overall, rpath gives us much more benefit than the occasional trouble it
creates.
--
Peter Eisentraut peter_e@gmx.net
On Sun, 18 Nov 2001, Peter Eisentraut wrote:
Trond Eivind Glomsr�d writes:
Fix? Don't use rpath - it's evil, and should be avoided anyway.
configure --disable-rpath is your friend.
Didn't know about that one - it will certainly be put to use :)
Overall, rpath gives us much more benefit than the occasional trouble it
creates.
When you install (not configure for, just install) into a separate tree
(for easier packaging), it's a hole which can be exploited - some packages
will rpath into /var/tmp/<foo>, for instance. Hackers can then put their
own library there. One big offender here is perl's automatic module
creation script which will change the rpaths from what you told it when
you built it to what you do when you install it.
--
Trond Eivind Glomsr�d
Red Hat, Inc.
Trond Eivind Glomsr�d writes:
When you install (not configure for, just install) into a separate tree
(for easier packaging), it's a hole which can be exploited - some packages
will rpath into /var/tmp/<foo>, for instance. Hackers can then put their
own library there.
"Some packages"... ;-)
One big offender here is perl's automatic module
creation script which will change the rpaths from what you told it when
you built it to what you do when you install it.
This should be fixed now, although the perl module will actually not obey
the --disable-rpath switch. Can't have everything, I guess...
--
Peter Eisentraut peter_e@gmx.net
Larry Rosenman <ler@lerctr.org> writes:
* Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]:
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
Since it works when prefix isn't /usr, I'd guess that the build
process sets rpath. This takes precedence over LD_LIBRARY_PATH.Fix? Don't use rpath - it's evil, and should be avoided anyway.
Peter, any thoughts on that? It's not clear to me that removing
rpath settings would be an improvement overall, even if it makes
"make check" more reliable.My general take is rpath is a *GOOD* thing, as it prevents USERS from
not being able to find REQUIRED shared libs.
Lib-directories should be listed in /etc/ld.so.conf, and no rpath used
- at least on linux.
--
Trond Eivind Glomsr�d
Red Hat, Inc.
* Trond Eivind Glomsr?d <teg@redhat.com> [011119 10:47]:
Larry Rosenman <ler@lerctr.org> writes:
* Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]:
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
Since it works when prefix isn't /usr, I'd guess that the build
process sets rpath. This takes precedence over LD_LIBRARY_PATH.Fix? Don't use rpath - it's evil, and should be avoided anyway.
Peter, any thoughts on that? It's not clear to me that removing
rpath settings would be an improvement overall, even if it makes
"make check" more reliable.My general take is rpath is a *GOOD* thing, as it prevents USERS from
not being able to find REQUIRED shared libs.Lib-directories should be listed in /etc/ld.so.conf, and no rpath used
- at least on linux.
no such animal on OpenUNIX 8 (In native "unix" mode).
--
Trond Eivind Glomsr�d
Red Hat, Inc.
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749