GUC with units, details

Peter Eisentraut <peter_e@gmx.net> writes:

Output from SHOW uses the largest unit that fits as long as the number
is an integer.

That seems OK for SHOW, which is mainly intended for human consumption,
but what will you do with pg_settings? For programmatic use I think
we want more predictable behavior.

I'm inclined to suggest adding a column "native units" to pg_settings,
which shows what the underlying units are (ie, the existing
interpretations) and then always show the value of a given variable
in its native unit.

regards, tom lane

On Jul 26, 2006, at 6:56 , Peter Eisentraut wrote:

Memory units are kB, MB, GB. The factor is 1024.

Time units are ms, s, min, h, d.

Are units case-sensitive? I've noticed you've been consistent in your
capitalization in these posts, so I'm wondering if you're enforcing
the same case in postgresql.conf.

Michael Glaesemann
grzm seespotcode net

On Jul 26, 2006, at 7:12 , Michael Glaesemann wrote:

On Jul 26, 2006, at 6:56 , Peter Eisentraut wrote:

Memory units are kB, MB, GB. The factor is 1024.

Time units are ms, s, min, h, d.

Are units case-sensitive? I've noticed you've been consistent in
your capitalization in these posts, so I'm wondering if you're
enforcing the same case in postgresql.conf.

Sorry for the noise. Didn't see that this was answered previously.

Michael Glaesemann
grzm seespotcode net

Peter Eisentraut wrote:

Memory units are kB, MB, GB. The factor is 1024.

Then shouldn't the factor be 1000? If the factor is 1024, then the units
should be KiB, MiB, GiB per IEEE 1541
(http://en.wikipedia.org/wiki/IEEE_1541) and others.

I'm not trying to be pedantic, but the general approach with -hackers
seems to be towards compliance where practical.

Regards,
Paul Bort

Bort, Paul wrote:

I'm not trying to be pedantic, but the general approach with -hackers
seems to be towards compliance where practical.

But in this case it's not practical.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e@gmx.net> writes:

Bort, Paul wrote:

[ 1000 vs 1024 ]

But in this case it's not practical.

Maybe I'm missing something, but I thought it was fairly common to use
"k" for 1000, "K" for 1024, etc (mnemonic: upper case for the larger
multiplier). So I'd vote for accepting "KB" rather than "kB" ...

regards, tom lane

On Tue, 2006-07-25 at 19:00 -0400, Tom Lane wrote:

Maybe I'm missing something, but I thought it was fairly common to use
"k" for 1000, "K" for 1024, etc (mnemonic: upper case for the larger
multiplier).

Well, that only works for K vs. k: the SI prefix for mega is M (meaning
10^6), not "m". Similarly for "G".

Why it is "impractical" to use the IEC prefixes?

-Neil

Neil Conway wrote:

On Tue, 2006-07-25 at 19:00 -0400, Tom Lane wrote:

Maybe I'm missing something, but I thought it was fairly common to
use "k" for 1000, "K" for 1024, etc (mnemonic: upper case for the
larger multiplier).

Well, that only works for K vs. k: the SI prefix for mega is M
(meaning 10^6), not "m". Similarly for "G".

Indeed. The k vs K idea is an excuse for not wanting to side with
either camp, but it does not scale.

Why it is "impractical" to use the IEC prefixes?

I'd imagine that one of the first things someone will want to try is
something like SET work_mem TO '10MB', which will fail or misbehave
because 10000000 bytes do not divide up into chunks of 1024 bytes. Who
wants to explain to users that they have to write '10MiB'?

Since about forever, PostgreSQL has used kB, MB, GB to describe memory
allocation. If we want to change that, we ought to do it across the
board. But that's a big board.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Wed, 2006-07-26 at 08:12 +0200, Peter Eisentraut wrote:

Neil Conway wrote:

On Tue, 2006-07-25 at 19:00 -0400, Tom Lane wrote:

Maybe I'm missing something, but I thought it was fairly common to
use "k" for 1000, "K" for 1024, etc (mnemonic: upper case for the
larger multiplier).

Well, that only works for K vs. k: the SI prefix for mega is M
(meaning 10^6), not "m". Similarly for "G".

Indeed. The k vs K idea is an excuse for not wanting to side with
either camp, but it does not scale.

Why it is "impractical" to use the IEC prefixes?

I'd imagine that one of the first things someone will want to try is
something like SET work_mem TO '10MB', which will fail or misbehave
because 10000000 bytes do not divide up into chunks of 1024 bytes. Who
wants to explain to users that they have to write '10MiB'?

Since about forever, PostgreSQL has used kB, MB, GB to describe memory
allocation. If we want to change that, we ought to do it across the
board. But that's a big board.

Neil is right: K, M, G are the correct SI terms, however, I don't see
any value in using that here. Nobody is suggesting we encourage or even
allow people to write max_fsm_pages = 10M rather than 10000000, so we
don't ever need to say that K = 1000, AFAICS. I think we are safe to
assume that

kB = KB = kb = Kb = 1024 bytes

mB = MB = mb = Mb = 1024 * 1024 bytes

gB = GB = gb = Gb = 1024 * 1024 * 1024 bytes

There's no value in forcing the use of specific case and it will be just
confusing for people.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

Simon Riggs wrote:

don't ever need to say that K = 1000, AFAICS. I think we are safe to
assume that

kB = KB = kb = Kb = 1024 bytes

mB = MB = mb = Mb = 1024 * 1024 bytes

gB = GB = gb = Gb = 1024 * 1024 * 1024 bytes

There's no value in forcing the use of specific case and it will be just
confusing for people.

It's fairly common to use 'b' for 'bits' and 'B' for 'bytes'. My suggestion would be to be
much more restrictive and avoid small caps:

KB = 1024 bytes
MB = 1024 KB
GB = 1024 KB
TB = 1024 GB

Although I don't expect to see bit-rates or fractions ('m' == 'milli') in GUC, it might be
good to use consistent units everywhere.

Regards,
Thomas Hallgren

Simon Riggs wrote:

There's no value in forcing the use of specific case and it will be
just confusing for people.

The issue was not the case of the units, but people were suggesting that
we should enforce the use of kiB, MiB, and GiB.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Tom Lane wrote:

That seems OK for SHOW, which is mainly intended for human
consumption, but what will you do with pg_settings? For programmatic
use I think we want more predictable behavior.

I'd think that a program would not care. Or do you want a units-free
display that can be parsed as integer?

Do we want to introduce a difference between pg_settings and SHOW ALL?
(Is there one already?)

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e@gmx.net> writes:

Tom Lane wrote:

That seems OK for SHOW, which is mainly intended for human
consumption, but what will you do with pg_settings? For programmatic
use I think we want more predictable behavior.

I'd think that a program would not care. Or do you want a units-free
display that can be parsed as integer?

Yeah. If the value might be shown as either "99kB" or "9MB" then a
program *must* have a pretty complete understanding of the units system
to make sense of it at all. Furthermore this is not backwards
compatible --- it'll break any existing code that inspects pg_settings
values. I suggest that the values column should continue to display
exactly as it does today (ie, the integer value in the var's native
units) and we just add a column saying what the native units are.

Do we want to introduce a difference between pg_settings and SHOW ALL?

Yup, I think that's the lesser of the evils.

regards, tom lane

Peter Eisentraut wrote:

I'd imagine that one of the first things someone will want to try is
something like SET work_mem TO '10MB', which will fail or misbehave
because 10000000 bytes do not divide up into chunks of 1024
bytes. Who
wants to explain to users that they have to write '10MiB'?

How about this:

INFO: Your setting was converted to IEC standard binary units. Use KiB,
MiB, and GiB to avoid this warning.

Since about forever, PostgreSQL has used kB, MB, GB to
describe memory
allocation. If we want to change that, we ought to do it across the
board. But that's a big board.

The standard hasn't been around forever; some incarnation of PostgreSQL
certainly pre-dates it. But it was created to reduce confusion between
binary and decimal units.

The Linux kernel changed to the standard years ago. And that's just a
few more lines of code than PostgreSQL. ( http://kerneltrap.org/node/340
and others )

Regards,
Paul Bort

Bort, Paul wrote:

The Linux kernel changed to the standard years ago. And that's just a
few more lines of code than PostgreSQL. (
http://kerneltrap.org/node/340 and others )

For your entertainment, here are the usage numbers from the linux-2.6.17
kernel:

kilobyte (-i) 82
kibibyte (-i) 2
megabyte (-i) 98
mebibyte (-i) 0
gigabyte (-i) 32
gibibyte (-i) 0

KB 1151
kB 407
KiB 181
MB 3830
MiB 298
GB 815
GiB 17

So I remain unconvinced.

Of course, your general point is a good one. If there are actually
systems using this, it might be worth considering. But if not, then
we're just going to confuse people.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut politely corrected:

For your entertainment, here are the usage numbers from the
linux-2.6.17
kernel:

kilobyte (-i) 82
kibibyte (-i) 2
megabyte (-i) 98
mebibyte (-i) 0
gigabyte (-i) 32
gibibyte (-i) 0

KB 1151
kB 407
KiB 181
MB 3830
MiB 298
GB 815
GiB 17

Thanks for the info. I had seen several articles on it, and shot my
mouth off without double-checking. My apologies.

I still think it would be a good idea to use the standard, and that this
is an opportunity to do so.

Regards,
Paul Bort

Peter Eisentraut wrote:

Bort, Paul wrote:

The Linux kernel changed to the standard years ago. And that's just a
few more lines of code than PostgreSQL. (
http://kerneltrap.org/node/340 and others )

For your entertainment, here are the usage numbers from the linux-2.6.17
kernel:

kilobyte (-i) 82
kibibyte (-i) 2
megabyte (-i) 98
mebibyte (-i) 0
gigabyte (-i) 32
gibibyte (-i) 0

KB 1151
kB 407
KiB 181
MB 3830
MiB 298
GB 815
GiB 17

So I remain unconvinced.

Of course, your general point is a good one. If there are actually
systems using this, it might be worth considering. But if not, then
we're just going to confuse people.

Is it worth bothering about the small deviation, if 10000 was meant, but
10k gives 10240 buffers? Isn't it quite common that systems round config
values to the next sensible value anyway?

Regards,
Andreas

On Wed, Jul 26, 2006 at 12:17:00PM -0400, Bort, Paul wrote:

Peter Eisentraut wrote:

I'd imagine that one of the first things someone will want to try is
something like SET work_mem TO '10MB', which will fail or misbehave
because 10000000 bytes do not divide up into chunks of 1024
bytes. Who
wants to explain to users that they have to write '10MiB'?

How about this:

INFO: Your setting was converted to IEC standard binary units. Use KiB,
MiB, and GiB to avoid this warning.

That's silly. If you're going to treat KB as 1024 bytes anyway,
complaining about it is just being pedantic.

The thing is, most memory sizes in postgres need to be some multiple of
a page size. You can't have a shared buffers of exactly 100000 bytes,
while 102400 bytes is possible. When someone has a GB of memory, they
really mean a GiB, but no-one bothers to correct them.

Is there anywhere in postgres where using K=1000 would be significantly
clearer than K=1024?

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Martijn van Oosterhout wrote:

How about this:

INFO: Your setting was converted to IEC standard binary

units. Use KiB,

MiB, and GiB to avoid this warning.

That's silly. If you're going to treat KB as 1024 bytes anyway,
complaining about it is just being pedantic.

But after a version or two with warnings, we have grounds to make it an
error. I'd rather just go with the standard from day 1 and reject
decimal units where they don't make sense, but that seems unlikely.

The thing is, most memory sizes in postgres need to be some
multiple of
a page size. You can't have a shared buffers of exactly 100000 bytes,
while 102400 bytes is possible. When someone has a GB of memory, they
really mean a GiB, but no-one bothers to correct them.

And hard drives are just the opposite: a 250GB drive does not have
268,435,456,000 bytes of unformatted space.

Is there anywhere in postgres where using K=1000 would be
significantly
clearer than K=1024?

If the unit for a setting is pages, then a value of '1K' could cause
some confusion as to whether that's 1,000 or 1,024 pages.