Re: [HACKERS] include/config.h FOLLOWUP
On Sat, 3 Jan 1998, Bruce Momjian wrote:
I believe it is 8k to match the base size for the filesystem block size,
for performance.Hrmmm...how does one find out what the file system block size is? I know
there is an option to newfs for changing this, and at least under FreeBSD, the
default is set to:sys/param.h:#define DFLTBSIZE 4096
So maybe a multiple of block size should be considered more appropriate?
Maybe have it so that you can stipulate # of blocks that equal max tuple size?
Then, if I wanted, I could format a drive with a block size of 16k that is only
going to be used for databases, and have a tuple size up to that level?
Yes, you certainly could do that. The comment says:
/*
* the maximum size of a disk block for any possible installation.
*
* in theory this could be anything, but in practice this is actually
* limited to 2^13 bytes because we have limited ItemIdData.lp_off and
* ItemIdData.lp_len to 13 bits (see itemid.h).
*/
#define MAXBLCKSZ 8192
You can now specify the actual file system block size at the time of
newfs. We actually could query the file system at time of compile, but
that would be strange becuase the database location is set at time of
postmaster startup, and I don't think we can make this a run-time
parameter, but I may be wrong.
Of course, you have to change the structures the mention.
--
Bruce Momjian
maillist@candle.pha.pa.us
Import Notes
Reply to msg id not found: Pine.NEB.3.96.980104004558.235U-100000@thelab.hub.org
On Sun, 4 Jan 1998, Bruce Momjian wrote:
Yes, you certainly could do that. The comment says:
/*
* the maximum size of a disk block for any possible installation.
*
* in theory this could be anything, but in practice this is actually
* limited to 2^13 bytes because we have limited ItemIdData.lp_off and
* ItemIdData.lp_len to 13 bits (see itemid.h).
*/
#define MAXBLCKSZ 8192You can now specify the actual file system block size at the time of
newfs. We actually could query the file system at time of compile, but
that would be strange becuase the database location is set at time of
postmaster startup, and I don't think we can make this a run-time
parameter, but I may be wrong.
No, don't make it a run-time or auto-detect thing, just a compile time
option. By default, leave it at 8192, since "that's the way its always been"...
but if we are justifying it based on disk block size, its 2x the disk block
size that my system is setup for. What's the difference between that and making
it 3x or 4x? Or, hell, would I get a performance increase if I brought it
down to 4096, which is what my actually disk block size is?
So, what we would really be doing is setting the default to 8192, but give
the installer the opportunity (with a caveat that this value should be a multiple
of default file system block size for optimal performance) to increase it as they
see fit.
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
No, don't make it a run-time or auto-detect thing, just a compile time
option. By default, leave it at 8192, since "that's the way its always been"...
but if we are justifying it based on disk block size, its 2x the disk block
size that my system is setup for. What's the difference between that and making
it 3x or 4x? Or, hell, would I get a performance increase if I brought it
down to 4096, which is what my actually disk block size is?So, what we would really be doing is setting the default to 8192, but give
the installer the opportunity (with a caveat that this value should be a multiple
of default file system block size for optimal performance) to increase it as they
see fit.
I assume you changed the default, becuase the BSD44 default is 8k
blocks, with 1k fragments.
I don't think there is any 'performance' improvement with making it
greater than the file system block size.
--
Bruce Momjian
maillist@candle.pha.pa.us
On Sun, 4 Jan 1998, Bruce Momjian wrote:
No, don't make it a run-time or auto-detect thing, just a compile time
option. By default, leave it at 8192, since "that's the way its always been"...
but if we are justifying it based on disk block size, its 2x the disk block
size that my system is setup for. What's the difference between that and making
it 3x or 4x? Or, hell, would I get a performance increase if I brought it
down to 4096, which is what my actually disk block size is?So, what we would really be doing is setting the default to 8192, but give
the installer the opportunity (with a caveat that this value should be a multiple
of default file system block size for optimal performance) to increase it as they
see fit.I assume you changed the default, becuase the BSD44 default is 8k
blocks, with 1k fragments.
Good question, I don't know. What does BSDi have it set at? Linux? NetBSD?
I just checked our sys/param.h file under Solaris 2.5.1, and it doesn't
seem to define a DEFAULT, but a MAXSIZE of 8192...oops, newfs defines the default
there for 8192 also
I don't think there is any 'performance' improvement with making it
greater than the file system block size.
No no...you missed the point. If we are saying that max tuple size is 8k
because of block size of the file system, under FreeBSD, the tuple size is 2x
the block size of the file system. So, if there a performance decrease because
of that...on modern OSs, how much does that even matter anymore? The 8192 that
we have current set, that's probably still from the original Postgres4.2 system
that was written in which decade? :)
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Sun, 4 Jan 1998, Bruce Momjian wrote:
No, don't make it a run-time or auto-detect thing, just a compile time
option. By default, leave it at 8192, since "that's the way its always been"...
but if we are justifying it based on disk block size, its 2x the disk block
size that my system is setup for. What's the difference between that and making
it 3x or 4x? Or, hell, would I get a performance increase if I brought it
down to 4096, which is what my actually disk block size is?So, what we would really be doing is setting the default to 8192, but give
the installer the opportunity (with a caveat that this value should be a multiple
of default file system block size for optimal performance) to increase it as they
see fit.I assume you changed the default, becuase the BSD44 default is 8k
blocks, with 1k fragments.Good question, I don't know. What does BSDi have it set at? Linux? NetBSD?
I just checked our sys/param.h file under Solaris 2.5.1, and it doesn't
seem to define a DEFAULT, but a MAXSIZE of 8192...oops, newfs defines the default
there for 8192 alsoI don't think there is any 'performance' improvement with making it
greater than the file system block size.No no...you missed the point. If we are saying that max tuple size is 8k
because of block size of the file system, under FreeBSD, the tuple size is 2x
the block size of the file system. So, if there a performance decrease because
of that...on modern OSs, how much does that even matter anymore? The 8192 that
we have current set, that's probably still from the original Postgres4.2 system
that was written in which decade? :)
I see, we could increase it and it probably would not matter much.
--
Bruce Momjian
maillist@candle.pha.pa.us
Integration wrote:
ps. why not allow for larger tuples in general? Do we take a speed hit?
Using large blocks is bad for performance: by increasing block size
you automatically decrease number of blocks in shared buffer pool -
this is bad for index scans and in multi-user environment!
Just remember that Informix (and others) use 2K blocks.
(Actually, I would like to have smaller blocks, but postgres lives
over file system...)
As for having big tuples - someone said about multi-representation
feature of Illustra (automatically storing of big fields outside
of tuple itself - in blobs, large objects, ...): looks very nice.
Vadim
Import Notes
Reference msg id not found: 199801040405.XAA15866@wilde.oit.umass.edu | Resolved by subject fallback
On Mon, 5 Jan 1998, Vadim B. Mikheev wrote:
Just remember that Informix (and others) use 2K blocks.
So we're 4x what the commercial ones are as of right now?
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Mon, 5 Jan 1998, Vadim B. Mikheev wrote:
Just remember that Informix (and others) use 2K blocks.
So we're 4x what the commercial ones are as of right now?
Date: Sat, 14 Dec 1996 17:29:53 -0500
From: aixssd!darrenk (Darren King)
To: abs.net!postgreSQL.org!hackers
Subject: [HACKERS] The 8k block size.
--- snip ---(All of this is taken from their respective web site docs)
block size max tuple size
IBM DB2 4K 4005 bytes
Sybase 2K 2016 bytes
Informix 2K 32767 bytes
Oracle (left my Oracle books at home...oops)
--- snip ---Darren darrenk@insightdist.com
Import Notes
Resolved by subject fallback
On Mon, 5 Jan 1998, Vadim B. Mikheev wrote:
Just remember that Informix (and others) use 2K blocks.
So we're 4x what the commercial ones are as of right now?
That is because they do not use the file system, so they try to match
the raw disk block sizes, while we try to match the file system size.
--
Bruce Momjian
maillist@candle.pha.pa.us
On Sun, 4 Jan 1998, Bruce Momjian wrote:
On Mon, 5 Jan 1998, Vadim B. Mikheev wrote:
Just remember that Informix (and others) use 2K blocks.
So we're 4x what the commercial ones are as of right now?
That is because they do not use the file system, so they try to match
the raw disk block sizes, while we try to match the file system size.
Irrelevant to my question...our tuples...are they 4x the size of the
commercial vendors, or is Vadim talking about something altogether different?
If we are 4x their size, then I think this whole discussion is a joke since
we are already *way* better then "the others"
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Sun, 4 Jan 1998, Bruce Momjian wrote:
On Mon, 5 Jan 1998, Vadim B. Mikheev wrote:
Just remember that Informix (and others) use 2K blocks.
So we're 4x what the commercial ones are as of right now?
That is because they do not use the file system, so they try to match
the raw disk block sizes, while we try to match the file system size.Irrelevant to my question...our tuples...are they 4x the size of the
commercial vendors, or is Vadim talking about something altogether different?If we are 4x their size, then I think this whole discussion is a joke since
we are already *way* better then "the others"
That's a good question. What is the maximum tuple size for Informix or
Oracle tuples?
--
Bruce Momjian
maillist@candle.pha.pa.us
On Mon, 5 Jan 1998, Vadim B. Mikheev wrote:
Just remember that Informix (and others) use 2K blocks.
So we're 4x what the commercial ones are as of right now?
(All of this is taken from their respective web site docs)
block size max tuple size
IBM DB2 4K 4005 bytes
Sybase 2K 2016 bytes
Informix 2K 32767 bytes
Oracle (left my Oracle books at home...oops)
Wow, I guess we are not as bad as I thought. If Peter gets large
objects working properly, we can close this issue.
--
Bruce Momjian
maillist@candle.pha.pa.us